-- Logs begin at Tue 2022-05-24 12:16:11 UTC, end at Tue 2022-05-24 13:55:23 UTC. -- May 24 12:44:39.410984 dv-cinder-843018-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 24 12:44:39.422122 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 24 12:44:39.424171 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 24 12:44:39 2022] *** May 24 12:44:39.424171 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 24 12:44:39.424171 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 24 12:44:39.424171 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: nodename: dv-cinder-843018-2-iscsi May 24 12:44:39.424171 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: machine: x86_64 May 24 12:44:39.424171 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: clock source: unix May 24 12:44:39.424171 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: pcre jit disabled May 24 12:44:39.424171 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: detected number of CPU cores: 4 May 24 12:44:39.424171 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: current working directory: / May 24 12:44:39.424171 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: detected binary path: /usr/bin/uwsgi-core May 24 12:44:39.424171 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: your processes number limit is 47680 May 24 12:44:39.424859 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: your memory page size is 4096 bytes May 24 12:44:39.424859 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: detected max file descriptor number: 2048 May 24 12:44:39.424859 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: lock engine: pthread robust mutexes May 24 12:44:39.424859 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: thunder lock: enabled May 24 12:44:39.424859 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 24 12:44:39.424859 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 24 12:44:39.468449 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: Python main interpreter initialized at 0x5614a93e39c0 May 24 12:44:39.468449 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: python threads support enabled May 24 12:44:39.468449 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: your server socket listen backlog is limited to 100 connections May 24 12:44:39.468449 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: your mercy for graceful operations on workers is 90 seconds May 24 12:44:39.468773 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: mapped 403077 bytes (393 KB) for 2 cores May 24 12:44:39.468855 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: *** Operational MODE: preforking *** May 24 12:44:39.469129 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: *** uWSGI is running in multiple interpreter mode *** May 24 12:44:39.469129 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: spawned uWSGI master process (pid: 80079) May 24 12:44:39.469300 dv-cinder-843018-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 24 12:44:39.470694 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: spawned uWSGI worker 1 (pid: 80090, cores: 1) May 24 12:44:39.470694 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: spawned uWSGI worker 2 (pid: 80091, cores: 1) May 24 12:44:39.470694 dv-cinder-843018-2-iscsi devstack@keystone.service[80079]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 24 12:44:45.052077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 12:44:45.052077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: from cryptography.utils import int_from_bytes May 24 12:44:45.052872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 12:44:45.052872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: from cryptography.utils import int_from_bytes May 24 12:44:45.082439 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 12:44:45.082439 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: from cryptography.utils import int_from_bytes May 24 12:44:45.082916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 12:44:45.082916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: from cryptography.utils import int_from_bytes May 24 12:44:45.100970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 12:44:45.101161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 12:44:45.101352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] command line args: [] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 12:44:45.101619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 12:44:45.101795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ================================================================================ {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 12:44:45.102046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] admin_token = **** {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.102314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] config_dir = [] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.102506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.102686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] config_source = [] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.102862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] debug = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.103057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.104419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.104419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.104419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.104419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.104419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] list_limit = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.104800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] log_config_append = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.104800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.104800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] log_dir = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.104990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] log_file = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.105176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.105402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.105648 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.105892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.106069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.106242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.106508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.106752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.106982 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.107212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.107439 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.107667 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.107911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.108153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.108426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.108904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.109084 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] publish_errors = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.109311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.109538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.109713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.109955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.110185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.110442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] standard_threads = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.110676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.110914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.111086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.111259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] use_journal = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.111432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] use_json = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.111603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] use_stderr = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.111774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] use_syslog = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.112002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.112255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.113022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.113281 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.113520 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.113767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.114003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.114249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.114506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.114750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.114995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.115245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.115477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.115721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.115961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.116206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.116452 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.116685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.116937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.117189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.117434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.117668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.117920 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.118178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.118451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.118698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.118928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.119155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.119383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.119616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.119861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.120106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.120334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.120562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.120828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.121050 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.121285 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.121532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.121762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.121993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.122224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.122473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.122700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.122926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.123152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.123385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.123633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.123860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.124201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.124434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.124670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.124905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.125151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.125379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.125614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.125847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.126085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.126340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.126574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.126800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.127047 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.127281 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.127514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.127744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.128567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.128804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.129034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.129268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.129501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.129767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.129851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 12:44:45.130002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 12:44:45.130080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.130199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] command line args: [] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 12:44:45.130278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.130500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.130580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 12:44:45.130680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ================================================================================ {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 12:44:45.130757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.130936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] admin_token = **** {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.131014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.131176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.131260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] config_dir = [] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.131364 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.131447 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.131547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] config_source = [] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.131631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.131732 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] debug = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.131862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.131943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.132102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.132182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.132336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.132415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.132568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.132646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.132800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.132878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.133032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.133111 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] list_limit = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.133210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] log_config_append = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.133286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.133389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.133509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.133598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] log_dir = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.133733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] log_file = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.133811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.133970 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.134049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.134198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.134272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.134448 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.134528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.134683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.134768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.134878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.134963 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.135057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.135136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.135305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.135389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.135553 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.135633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.135779 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.135855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.136011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.136094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.136236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.136314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.136471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.136547 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.136701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.136777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.136946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.137033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.137179 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.137268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.137432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.137945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.138044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.138119 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] publish_errors = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.138188 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.138364 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.138462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] auth.external = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.138598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.138676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.138777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.138923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.139048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.139147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.139243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.139369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] auth.password = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.139474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] standard_threads = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.139598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] auth.token = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.139711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.139838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.139973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.140061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.140189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.140265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] use_journal = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.140390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.140512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] use_json = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.140595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.140668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] use_stderr = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.140784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] use_syslog = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.140858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.140982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 12:44:45.141058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.141198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.141275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.141430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] credential.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.141552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.141675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.141762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.141918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.142003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.142161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.142239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.142418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.142500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.142661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.142741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.142893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.142971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.143142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.143231 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.143391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.143469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.143649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.143805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.143886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.143968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.144165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.144287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.144403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.144481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.144650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.144782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.144889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.144967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.145126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.145209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.145360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] federation.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.145446 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.145595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.145674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.145837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.145923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.146084 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.146170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.146330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.146422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.146580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.146660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.146813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.146971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.147047 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.147118 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.147284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.147367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.147527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.147616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.147758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.147842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.147994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.148081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.148224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.148310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.148468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.148546 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.148700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.148778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.148931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.149018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.149165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.149256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.149408 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.149487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.149635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.149713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.149869 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.150003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.150108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.150193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.150367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.150448 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.150605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.150686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.150832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.150910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.151060 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.151138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.151305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.151385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.151536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.151620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.151771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.151863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.152020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.152099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.152252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.152340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.152488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.152567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.152720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.152838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.152976 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.153104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.153182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.153253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.153430 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.153510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.153661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.153741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.153898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.153985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.154135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.154222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.154394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.154478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.154632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.154713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.154869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.154951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.155081 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.155160 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.155336 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.155425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.155569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.155647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.155802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.155882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.156034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.156127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.156266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.156345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.156501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.156582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.156727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.156812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.156962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.157040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.157193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.157272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.157430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.157513 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.157664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.157755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.157898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.157979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.158144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.158223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.158411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.158494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.158641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.158768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.158867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.158947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.159110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.159189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.159375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.159455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.159609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.159686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.159843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.159922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.160067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.160144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.160290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.160371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.160526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.160605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.160761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.160841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.160997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.161075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.161239 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.161319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.161482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.161562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.161721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.161799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.161952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.162030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.162179 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.162260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.162435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.162515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.162666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.162746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.162899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.162980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.163125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.163204 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.163355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.163434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.163587 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.163668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.163825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.163904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.164056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.164144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.164297 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.164377 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.164521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.164598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.164763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.164850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.164996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.165083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.165229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.165310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.165471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.165559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.165703 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.165783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.165945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] auth.external = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.166030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.166177 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.166266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.166437 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.166517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.166690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.166769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.166919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.167011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] auth.password = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.167156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] auth.token = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.167235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.167385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.167464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.167620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.167709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.167855 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.167936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.168085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.168164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.168326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.168413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.168556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.168644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.168788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.168867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.169024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] credential.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.169112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] resource.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.169276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.169394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.169521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.169601 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.169753 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.169843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.169999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.170078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.170242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.170349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.170497 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.170589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.170742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.170821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.170976 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.171054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.171212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.171303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.171445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] role.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.171524 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.171689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] role.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.171766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.171917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.172003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.172161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.172239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.172424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.172511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.172645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.172741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.172888 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.172967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] federation.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.173130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.173217 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.173406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.173492 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.173658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.173737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.173893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.173972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.174144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.174232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.174408 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.174489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.174654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.174731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.175403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.175484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.175648 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.175725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.175888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.176053 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.176151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.176277 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.176355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.176524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.176602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.176772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.176857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.177019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.177098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.177267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.177354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.177502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.177589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.177739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.177817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.177978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.178056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.178217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.178327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.178471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.178549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.178702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.178780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.178932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.179018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.179175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.179253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.179422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.179501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.179668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.179753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.179906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.179984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.180140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.180219 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.180369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.180455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.180604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] token.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.180682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.180839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.180917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.181085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.181175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.181320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.181398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.181555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.181632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.181787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.181875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.182025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.182104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.182265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.182360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.182527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.182617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.182771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.182848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.183001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.183077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.183225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.183299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.183464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.183542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.183708 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.183785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.183953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.184032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.184178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.184255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.184419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.184498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.184655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.184734 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.184940 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.185025 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.connection = **** {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.185181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.185269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.185421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.185500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.185655 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.185733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.185891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.185968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.186124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.186201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.186392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.186473 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.186630 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.186708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.186863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.186939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.187095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.187176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.187335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.187418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.187575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.187653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.187808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.187888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.188045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.188121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.188276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.188352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.188505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.188584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.188733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.188809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.188972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.189050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.189205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.189285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.189356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 12:44:45.189452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.189701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.189952 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.190188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.190463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.191244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.191481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.191720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.192050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.192308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.192561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.192816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.193069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.193321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.193554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.193803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.194045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.194300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.194547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.194787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.195035 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.195286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.195527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.195784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.196019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.196266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.196505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.196744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.196985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.197235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.197491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.197727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.197968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.198212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] resource.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.198497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.198752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.198992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.199243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.199494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.199735 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.199985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.200226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.200465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.200701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] role.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.200953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] role.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.201187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.201437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.201684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.201933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.202184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.202464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.202713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.202962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.203212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.203449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.203697 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.203947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.204196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.204445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.204838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.205076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.205328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.205580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.205831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.206114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.206377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.206639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.206894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.207134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.207369 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.207603 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.207835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.208089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.208346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.208596 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.208835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.209071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.209304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.209541 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] token.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.209784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.210034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.210274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.210553 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.210803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.211042 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.211287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.211532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.211780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.212017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.212257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.212491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.212741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.212990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.213230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.213474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.213774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.214036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.connection = **** {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.214298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.214551 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.214791 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.215033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.215271 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.215508 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.215746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.215983 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.216242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.216479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.216724 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.216965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.217216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.217454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.217701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.217934 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.218178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.218444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 12:44:45.218617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 12:44:45.220129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.220363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.220589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.229578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.244129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.244366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.244598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.254467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.258191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.258450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.273176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.273409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.311733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.316462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.362816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.363045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.363596 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.363835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.429625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.429860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.429965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.430067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.430147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.430295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.430418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.430531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.430633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.430748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.430830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.430959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.454419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.454656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.454741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.454887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.454969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.455115 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.455198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.455329 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.455410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.455541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.455622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.455750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.455839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.455961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.467554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.467811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.480117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.480297 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.480390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.480553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.480637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.480775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.480867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.480987 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.481069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.481198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.481280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.481407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.481497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.481616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 12:44:45.498301 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.498514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 12:44:45.498850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.498979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.499064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 12:44:45.499187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 12:44:45.499310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.499523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.499639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.499735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 12:44:45.499815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.499970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.500064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.500228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.500479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.500729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.512163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.512434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.512948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.513222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.513481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.513757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.514023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.514323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.521303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.522444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.524541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.524813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.525080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.525354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.525627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.525710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.525879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.525961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.526138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.526219 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.526514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.526598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.526785 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.526865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.527054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.527148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.527314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.527396 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.527667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.527938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.528216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.528487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.553856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.555167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.555469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.555850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.556876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.557155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.561035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.561312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.561588 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.562444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.562684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.562967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.569391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.569675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.570745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.571027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.574080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.574387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.574664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.575045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.575315 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.575489 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.575573 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.575840 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.575920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.576104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.576187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.576419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.576695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.576969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.577237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.577501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.594691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.594962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.595947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.596224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.599702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.600908 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.603306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.603593 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.604556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.604849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.608699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.609820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.612826 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.614249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.616400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.616694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.617086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.617379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.617655 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.617932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.618016 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.618311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.618603 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.618888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.619182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.619467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.637150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.637426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.637942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.638200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.639075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.639357 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.639623 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.639893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.646168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.647880 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.647967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.648151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.649594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.649868 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.651178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.651452 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.651727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.652004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.652283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.652912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.653192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.653472 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.653771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.654028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.665544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.665820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.666103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.666406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.666683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.666952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.667229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.667454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.667733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.668014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.668303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.668583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.668863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.669138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.685928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.688057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.689267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.691405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.692597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.694248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.694706 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.695893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.696174 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.696319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.696451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.697919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.698205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.698535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.703869 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.705456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.705884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.708063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.709173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.709463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.709745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.710015 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.711283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.711571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.711857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.712141 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.719022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.719302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.719576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.721202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.721484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.721915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.727734 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.728008 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.728282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.728551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.728822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.729088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.729376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.729640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.729722 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 12:44:45.729909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.729989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.730169 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.730258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.730458 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.730540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.730716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.730849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.730981 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.731061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.731249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.731330 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.731580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.731857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.732141 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.732409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.732668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.732929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.733195 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 12:44:45.770762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5614a93e39c0 pid: 80090 (default app) May 24 12:44:45.772703 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5614a93e39c0 pid: 80091 (default app) May 24 12:44:45.773937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd6b48-0f57-4501-8c5f-25948378da19 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:45.774228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd6b48-0f57-4501-8c5f-25948378da19 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:45.774527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd6b48-0f57-4501-8c5f-25948378da19 None None] PATH_INFO: `/v3` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:45.778769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Tue May 24 12:44:45 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:44:53.723169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984c418-66d3-4c91-8a2b-7ff54f3603de None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:53.723688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984c418-66d3-4c91-8a2b-7ff54f3603de None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:53.723688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984c418-66d3-4c91-8a2b-7ff54f3603de None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:53.728068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1/2] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:44:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:44:53.733054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5e345-a4fa-4b3b-9166-0c33eaa48cd9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:53.733428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5e345-a4fa-4b3b-9166-0c33eaa48cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:53.733614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5e345-a4fa-4b3b-9166-0c33eaa48cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:53.801657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: INFO dbcounter [None req-37a5e345-a4fa-4b3b-9166-0c33eaa48cd9 None None] Registered counter for database keystone May 24 12:44:53.822429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG oslo_db.sqlalchemy.engines [None req-37a5e345-a4fa-4b3b-9166-0c33eaa48cd9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80090) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 12:44:53.828603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writer thread running {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 12:44:54.008038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG passlib.handlers.bcrypt [None req-37a5e345-a4fa-4b3b-9166-0c33eaa48cd9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80090) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 24 12:44:54.008384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG passlib.handlers.bcrypt [None req-37a5e345-a4fa-4b3b-9166-0c33eaa48cd9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80090) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 24 12:44:54.063122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-37a5e345-a4fa-4b3b-9166-0c33eaa48cd9 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:44:54.064755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-37a5e345-a4fa-4b3b-9166-0c33eaa48cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:44:54.188049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 2/3] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:44:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 24 12:44:54.360512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984c418-66d3-4c91-8a2b-7ff54f3603de None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:44:54.429964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: INFO dbcounter [None req-2984c418-66d3-4c91-8a2b-7ff54f3603de None None] Registered counter for database keystone May 24 12:44:54.449146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG oslo_db.sqlalchemy.engines [None req-2984c418-66d3-4c91-8a2b-7ff54f3603de None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80091) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 12:44:54.455207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writer thread running {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 12:44:54.574176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984c418-66d3-4c91-8a2b-7ff54f3603de None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:44:54.578430 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f148f66-2e48-48a3-825b-38d64ed26815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:44:54.579093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148f66-2e48-48a3-825b-38d64ed26815 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:54.579379 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148f66-2e48-48a3-825b-38d64ed26815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:54.579658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148f66-2e48-48a3-825b-38d64ed26815 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:54.592856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f148f66-2e48-48a3-825b-38d64ed26815 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:44:54.644092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9f148f66-2e48-48a3-825b-38d64ed26815 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 24 12:44:54.644381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f148f66-2e48-48a3-825b-38d64ed26815 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:44:54.653923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 2/4] 10.222.0.45 () {58 vars in 1324 bytes} [Tue May 24 12:44:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 12:44:54.659985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5e345-a4fa-4b3b-9166-0c33eaa48cd9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:44:54.674051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5e345-a4fa-4b3b-9166-0c33eaa48cd9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:44:54.679046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8352094-b3e9-40e8-b4d1-79ab7fd3e167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:44:54.680153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8352094-b3e9-40e8-b4d1-79ab7fd3e167 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:54.680435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8352094-b3e9-40e8-b4d1-79ab7fd3e167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:54.680714 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8352094-b3e9-40e8-b4d1-79ab7fd3e167 admin admin] PATH_INFO: `/v3/projects/26aa7dae9c0c4f5a892a6fcef95c15ea` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:54.682194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8352094-b3e9-40e8-b4d1-79ab7fd3e167 admin admin] RBAC: Authorizing `identity:get_project(project_id=26aa7dae9c0c4f5a892a6fcef95c15ea)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:44:54.839488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e8352094-b3e9-40e8-b4d1-79ab7fd3e167 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 24 12:44:54.840137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8352094-b3e9-40e8-b4d1-79ab7fd3e167 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:44:54.844442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 3/5] 10.222.0.45 () {56 vars in 1207 bytes} [Tue May 24 12:44:54 2022] GET /identity/v3/projects/26aa7dae9c0c4f5a892a6fcef95c15ea => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:44:57.010830 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb28818-b0e9-4061-af2d-34ef35830d79 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:57.011348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb28818-b0e9-4061-af2d-34ef35830d79 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:57.011431 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb28818-b0e9-4061-af2d-34ef35830d79 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:57.015740 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 3/6] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:44:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:44:57.021768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917a12-f978-45e9-886c-ff7dc6dff1d5 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:57.022164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917a12-f978-45e9-886c-ff7dc6dff1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:57.022354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917a12-f978-45e9-886c-ff7dc6dff1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:57.041388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-15917a12-f978-45e9-886c-ff7dc6dff1d5 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:44:57.042918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-15917a12-f978-45e9-886c-ff7dc6dff1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:44:57.050575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 4/7] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:44:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 24 12:44:57.071497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb28818-b0e9-4061-af2d-34ef35830d79 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:44:57.083731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb28818-b0e9-4061-af2d-34ef35830d79 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:44:57.087893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8c2b-c0e0-4e8d-aca3-421975006dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:44:57.088523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8c2b-c0e0-4e8d-aca3-421975006dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:57.088810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8c2b-c0e0-4e8d-aca3-421975006dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:57.089090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8c2b-c0e0-4e8d-aca3-421975006dcc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:57.097244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb8c2b-c0e0-4e8d-aca3-421975006dcc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:44:57.099045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb8c2b-c0e0-4e8d-aca3-421975006dcc admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:44:57.107621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 4/8] 10.222.0.45 () {58 vars in 1324 bytes} [Tue May 24 12:44:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 12:44:57.112654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15917a12-f978-45e9-886c-ff7dc6dff1d5 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:44:57.121367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15917a12-f978-45e9-886c-ff7dc6dff1d5 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:44:57.125626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce46a2b-1530-4446-a156-7a1672e841c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:44:57.126495 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce46a2b-1530-4446-a156-7a1672e841c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:57.126778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce46a2b-1530-4446-a156-7a1672e841c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:57.127052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce46a2b-1530-4446-a156-7a1672e841c1 admin admin] PATH_INFO: `/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:57.128461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce46a2b-1530-4446-a156-7a1672e841c1 admin admin] RBAC: Authorizing `identity:get_user(user_id=c0168c20cbe64bb7a4d17219c4fe3eef)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:44:57.130272 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce46a2b-1530-4446-a156-7a1672e841c1 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:44:57.131828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 5/9] 10.222.0.45 () {56 vars in 1198 bytes} [Tue May 24 12:44:57 2022] GET /identity/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:44:59.357723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137ab0-db4d-40c5-807b-d4863f3d1b0a None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:59.358243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137ab0-db4d-40c5-807b-d4863f3d1b0a None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:59.358404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137ab0-db4d-40c5-807b-d4863f3d1b0a None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:59.359769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 5/10] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:44:59.365438 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1b1d4-8cac-4ad8-a3f1-7fbbae0a3f20 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:59.365816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1b1d4-8cac-4ad8-a3f1-7fbbae0a3f20 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:59.366001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1b1d4-8cac-4ad8-a3f1-7fbbae0a3f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:59.382993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8fb1b1d4-8cac-4ad8-a3f1-7fbbae0a3f20 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:44:59.384186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8fb1b1d4-8cac-4ad8-a3f1-7fbbae0a3f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:44:59.388210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7b6b-fc74-4580-b0cf-f27a418aa423 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:59.388496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7b6b-fc74-4580-b0cf-f27a418aa423 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:59.388773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7b6b-fc74-4580-b0cf-f27a418aa423 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:59.390112 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 6/11] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:44:59.395656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d78ddb-76b5-4ace-bfb4-1d6592dcffaf None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:59.395947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d78ddb-76b5-4ace-bfb4-1d6592dcffaf None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:59.397559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d78ddb-76b5-4ace-bfb4-1d6592dcffaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:59.435633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG passlib.handlers.bcrypt [None req-04d78ddb-76b5-4ace-bfb4-1d6592dcffaf None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80091) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 24 12:44:59.436053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG passlib.handlers.bcrypt [None req-04d78ddb-76b5-4ace-bfb4-1d6592dcffaf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80091) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 24 12:44:59.466915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 6/12] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:44:59.482929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-04d78ddb-76b5-4ace-bfb4-1d6592dcffaf None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:44:59.484516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-04d78ddb-76b5-4ace-bfb4-1d6592dcffaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:44:59.490550 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1b1d4-8cac-4ad8-a3f1-7fbbae0a3f20 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:44:59.513340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:44:59.580879 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 7/13] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:44:59.593229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1b1d4-8cac-4ad8-a3f1-7fbbae0a3f20 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:44:59.602916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d78ddb-76b5-4ace-bfb4-1d6592dcffaf None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:44:59.626316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfc006-7c7c-4a50-8b7e-2191afb622cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:44:59.627064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfc006-7c7c-4a50-8b7e-2191afb622cb None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:59.627353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfc006-7c7c-4a50-8b7e-2191afb622cb None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:59.628157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfc006-7c7c-4a50-8b7e-2191afb622cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:59.639823 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfc006-7c7c-4a50-8b7e-2191afb622cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-0cbfc006-7c7c-4a50-8b7e-2191afb622cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:44:59.651198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:44:59.652287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:44:59.653211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 7/14] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:44:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:44:59.658124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfc006-7c7c-4a50-8b7e-2191afb622cb None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:44:59.702734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d78ddb-76b5-4ace-bfb4-1d6592dcffaf None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:44:59.705125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:44:59.739965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f29533-2854-4b4e-bb32-26d2f02242a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:44:59.740865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f29533-2854-4b4e-bb32-26d2f02242a6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:59.741164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f29533-2854-4b4e-bb32-26d2f02242a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:59.741444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f29533-2854-4b4e-bb32-26d2f02242a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:59.745459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:44:59.765114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfc006-7c7c-4a50-8b7e-2191afb622cb None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:44:59.798461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b762a-bce9-4c81-a2c3-dc447158a6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:44:59.799127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b762a-bce9-4c81-a2c3-dc447158a6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:59.799431 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b762a-bce9-4c81-a2c3-dc447158a6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:59.799715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b762a-bce9-4c81-a2c3-dc447158a6f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:59.800471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b762a-bce9-4c81-a2c3-dc447158a6f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:44:59.800778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b762a-bce9-4c81-a2c3-dc447158a6f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:44:59.802174 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b762a-bce9-4c81-a2c3-dc447158a6f8 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:44:59.806961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f29533-2854-4b4e-bb32-26d2f02242a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:44:59.808462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f29533-2854-4b4e-bb32-26d2f02242a6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:44:59.813624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 8/15] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:44:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:44:59.819556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b762a-bce9-4c81-a2c3-dc447158a6f8 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:44:59.906321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 8/16] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:44:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:44:59.912216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f29533-2854-4b4e-bb32-26d2f02242a6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:44:59.914847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:44:59.917966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b762a-bce9-4c81-a2c3-dc447158a6f8 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:44:59.950502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebce7d-0b02-4897-884c-4eee1cea89a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:44:59.951374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebce7d-0b02-4897-884c-4eee1cea89a9 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:44:59.951654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebce7d-0b02-4897-884c-4eee1cea89a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:44:59.951930 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebce7d-0b02-4897-884c-4eee1cea89a9 None admin] PATH_INFO: `/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:44:59.953126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebce7d-0b02-4897-884c-4eee1cea89a9 None admin] RBAC: Authorizing `identity:get_user(user_id=c0168c20cbe64bb7a4d17219c4fe3eef)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:44:59.954630 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebce7d-0b02-4897-884c-4eee1cea89a9 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:44:59.956747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 9/17] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:44:59 2022] GET /identity/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:44:59.962093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebce7d-0b02-4897-884c-4eee1cea89a9 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:44:59.976396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:00.011155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f29533-2854-4b4e-bb32-26d2f02242a6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:00.045046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33962ee-24b6-4228-a8e7-98cb5b32b0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:00.045863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33962ee-24b6-4228-a8e7-98cb5b32b0dd None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:00.046142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33962ee-24b6-4228-a8e7-98cb5b32b0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:00.046449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33962ee-24b6-4228-a8e7-98cb5b32b0dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:00.061989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebce7d-0b02-4897-884c-4eee1cea89a9 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:00.064492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33962ee-24b6-4228-a8e7-98cb5b32b0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:00.065993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33962ee-24b6-4228-a8e7-98cb5b32b0dd None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-d33962ee-24b6-4228-a8e7-98cb5b32b0dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:45:00.086820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:00.087771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:00.088760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:45:00.088760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:45:00.088760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:45:00.088760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:00.088760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 9/18] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:00.094022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33962ee-24b6-4228-a8e7-98cb5b32b0dd None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:00.098255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794009c1-5397-470d-90aa-6ed802f285fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:00.099075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794009c1-5397-470d-90aa-6ed802f285fd None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:00.099358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794009c1-5397-470d-90aa-6ed802f285fd None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:00.099633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794009c1-5397-470d-90aa-6ed802f285fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:00.100811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794009c1-5397-470d-90aa-6ed802f285fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:00.102252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794009c1-5397-470d-90aa-6ed802f285fd None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:00.103844 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 10/19] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:44:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:00.109271 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794009c1-5397-470d-90aa-6ed802f285fd None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:00.155418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:00.192833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:00.199324 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33962ee-24b6-4228-a8e7-98cb5b32b0dd None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:00.212239 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794009c1-5397-470d-90aa-6ed802f285fd None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:00.231928 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e0e35-6513-4d12-9bec-faa2aa4b9d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:00.232592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e0e35-6513-4d12-9bec-faa2aa4b9d7f None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:00.232878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e0e35-6513-4d12-9bec-faa2aa4b9d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:00.233157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e0e35-6513-4d12-9bec-faa2aa4b9d7f None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:00.233880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e0e35-6513-4d12-9bec-faa2aa4b9d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:00.234187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e0e35-6513-4d12-9bec-faa2aa4b9d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:00.235595 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e0e35-6513-4d12-9bec-faa2aa4b9d7f None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:00.246302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b5093-431d-49fb-89ac-457a227c0608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:00.246967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5093-431d-49fb-89ac-457a227c0608 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:00.247251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5093-431d-49fb-89ac-457a227c0608 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:00.247525 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5093-431d-49fb-89ac-457a227c0608 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:00.248346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b5093-431d-49fb-89ac-457a227c0608 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.domain.id=default, user.id=c0168c20cbe64bb7a4d17219c4fe3eef) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:00.248659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b5093-431d-49fb-89ac-457a227c0608 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:00.250098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b5093-431d-49fb-89ac-457a227c0608 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:00.254922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 10/20] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:00.258877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 11/21] 10.222.0.45 () {56 vars in 1322 bytes} [Tue May 24 12:45:00 2022] GET /identity/v3/role_assignments?user.id=c0168c20cbe64bb7a4d17219c4fe3eef&scope.domain.id=default&role.id=0c473678b281495f86f58b077be04e1b => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:02.880388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af304a11-c26b-42d2-a4c7-a63db919cd46 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:02.881444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af304a11-c26b-42d2-a4c7-a63db919cd46 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:02.881940 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af304a11-c26b-42d2-a4c7-a63db919cd46 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:02.883599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 11/22] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:02.889521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae13fad-3c01-4866-83dd-eaa7d1181a20 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:02.890363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae13fad-3c01-4866-83dd-eaa7d1181a20 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:02.890879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae13fad-3c01-4866-83dd-eaa7d1181a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:02.898163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651815-2910-415e-a569-6054e1c278b2 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:02.898729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651815-2910-415e-a569-6054e1c278b2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:02.899213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651815-2910-415e-a569-6054e1c278b2 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:02.901908 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 12/23] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:02.906855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c882f-4373-4895-8911-ad8f20e6f477 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:02.907417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c882f-4373-4895-8911-ad8f20e6f477 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:02.907903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c882f-4373-4895-8911-ad8f20e6f477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:02.923292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6ae13fad-3c01-4866-83dd-eaa7d1181a20 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:02.925169 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6ae13fad-3c01-4866-83dd-eaa7d1181a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:02.929553 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-133c882f-4373-4895-8911-ad8f20e6f477 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:02.930999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-133c882f-4373-4895-8911-ad8f20e6f477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:03.036788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 12/24] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:03.038090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e1312-8eda-4f9f-8f83-b4b2089e6553 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:03.038416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e1312-8eda-4f9f-8f83-b4b2089e6553 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:03.038704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e1312-8eda-4f9f-8f83-b4b2089e6553 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:03.041417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 13/25] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:03.056424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134e0cd-b7c6-4927-bfe3-a695b2d6fd0c None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:03.056718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134e0cd-b7c6-4927-bfe3-a695b2d6fd0c None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:03.057176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134e0cd-b7c6-4927-bfe3-a695b2d6fd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:03.086517 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8134e0cd-b7c6-4927-bfe3-a695b2d6fd0c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:03.088048 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8134e0cd-b7c6-4927-bfe3-a695b2d6fd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:03.181014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 13/26] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:03.181594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c882f-4373-4895-8911-ad8f20e6f477 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:03.187036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 14/27] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:03.216943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134e0cd-b7c6-4927-bfe3-a695b2d6fd0c None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:03.223507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:03.233346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:03.295822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c882f-4373-4895-8911-ad8f20e6f477 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:03.334395 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4ed44-c25c-47ac-9673-94985b81be8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:03.335433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4ed44-c25c-47ac-9673-94985b81be8f None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:03.335856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4ed44-c25c-47ac-9673-94985b81be8f None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:03.336304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4ed44-c25c-47ac-9673-94985b81be8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:03.360646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4ed44-c25c-47ac-9673-94985b81be8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-bcd4ed44-c25c-47ac-9673-94985b81be8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:03.370335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:03.371239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:03.371994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:03.374742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 14/28] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:45:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:03.375710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4ed44-c25c-47ac-9673-94985b81be8f None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:03.398087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134e0cd-b7c6-4927-bfe3-a695b2d6fd0c None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:03.481043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4ed44-c25c-47ac-9673-94985b81be8f None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:03.483892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b6db9-20bc-408f-9bec-a80f30c0c43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:03.485028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b6db9-20bc-408f-9bec-a80f30c0c43e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:03.485518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b6db9-20bc-408f-9bec-a80f30c0c43e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:03.485995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b6db9-20bc-408f-9bec-a80f30c0c43e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:03.492474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:03.510752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b6db9-20bc-408f-9bec-a80f30c0c43e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:03.512244 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b6db9-20bc-408f-9bec-a80f30c0c43e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:03.527909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c353b7-218f-416a-b1cb-3e1994ad92de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:03.528612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c353b7-218f-416a-b1cb-3e1994ad92de None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:03.528895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c353b7-218f-416a-b1cb-3e1994ad92de None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:03.529173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c353b7-218f-416a-b1cb-3e1994ad92de None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:03.529953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c353b7-218f-416a-b1cb-3e1994ad92de None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:03.531446 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c353b7-218f-416a-b1cb-3e1994ad92de None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-5f1b6db9-20bc-408f-9bec-a80f30c0c43e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:45:03.550220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:03.551418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:03.552649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:45:03.552649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:45:03.552649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:45:03.552649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:03.554453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 15/29] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:03.555471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b6db9-20bc-408f-9bec-a80f30c0c43e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:03.672719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:03.753765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b6db9-20bc-408f-9bec-a80f30c0c43e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:03.758972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 15/30] 10.222.0.45 () {60 vars in 1131 bytes} [Tue May 24 12:45:03 2022] POST /identity/v3/roles => generated 219 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:03.760116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c353b7-218f-416a-b1cb-3e1994ad92de None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:03.790404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea48e9-6054-4893-b7ae-1b36d27b0c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:03.791109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea48e9-6054-4893-b7ae-1b36d27b0c30 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:03.791497 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea48e9-6054-4893-b7ae-1b36d27b0c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:03.791672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea48e9-6054-4893-b7ae-1b36d27b0c30 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:03.792445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea48e9-6054-4893-b7ae-1b36d27b0c30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:03.792757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea48e9-6054-4893-b7ae-1b36d27b0c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:03.794434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea48e9-6054-4893-b7ae-1b36d27b0c30 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:03.804360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 16/31] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:03.810162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fbbc3-1f24-492b-8e4c-172cffafdb36 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:03.810162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fbbc3-1f24-492b-8e4c-172cffafdb36 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:03.810162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fbbc3-1f24-492b-8e4c-172cffafdb36 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:03.810162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 17/32] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:03.811546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fbbc3-1f24-492b-8e4c-172cffafdb36 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:03.879742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c353b7-218f-416a-b1cb-3e1994ad92de None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:03.927529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3465254-e2ab-46ac-b94a-55f2bbf40a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:03.928946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3465254-e2ab-46ac-b94a-55f2bbf40a88 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:03.929407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3465254-e2ab-46ac-b94a-55f2bbf40a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:03.929856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3465254-e2ab-46ac-b94a-55f2bbf40a88 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:03.930836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3465254-e2ab-46ac-b94a-55f2bbf40a88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:03.931333 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3465254-e2ab-46ac-b94a-55f2bbf40a88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:03.933179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3465254-e2ab-46ac-b94a-55f2bbf40a88 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:03.946196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 16/33] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:03.948020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2301e-8861-4da3-b5e2-ac6314f6563c None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:03.948487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2301e-8861-4da3-b5e2-ac6314f6563c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:03.948971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2301e-8861-4da3-b5e2-ac6314f6563c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:03.961795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fbbc3-1f24-492b-8e4c-172cffafdb36 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:03.967909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5ea2301e-8861-4da3-b5e2-ac6314f6563c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:03.969690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5ea2301e-8861-4da3-b5e2-ac6314f6563c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:03.993890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:03.999202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:04.035613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8dd0e8-d32e-48ac-9cd2-47d0137d779b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:04.036525 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dd0e8-d32e-48ac-9cd2-47d0137d779b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:04.036812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dd0e8-d32e-48ac-9cd2-47d0137d779b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:04.037092 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dd0e8-d32e-48ac-9cd2-47d0137d779b None admin] PATH_INFO: `/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:04.038396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8dd0e8-d32e-48ac-9cd2-47d0137d779b None admin] RBAC: Authorizing `identity:get_user(user_id=c0168c20cbe64bb7a4d17219c4fe3eef)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:04.039988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8dd0e8-d32e-48ac-9cd2-47d0137d779b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:04.052073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 18/34] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:03 2022] GET /identity/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:04.052073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8dd0e8-d32e-48ac-9cd2-47d0137d779b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:04.054590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 17/35] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:04.055788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2301e-8861-4da3-b5e2-ac6314f6563c None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:04.173827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2301e-8861-4da3-b5e2-ac6314f6563c None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:04.210905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d93b0-9cd1-4cf3-9388-78b5e7a1463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:04.212157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d93b0-9cd1-4cf3-9388-78b5e7a1463c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:04.212656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d93b0-9cd1-4cf3-9388-78b5e7a1463c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:04.213149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d93b0-9cd1-4cf3-9388-78b5e7a1463c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:04.214935 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d93b0-9cd1-4cf3-9388-78b5e7a1463c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:04.216726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d93b0-9cd1-4cf3-9388-78b5e7a1463c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:04.218789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 18/36] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:04.219729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d93b0-9cd1-4cf3-9388-78b5e7a1463c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:04.236288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8dd0e8-d32e-48ac-9cd2-47d0137d779b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:04.248107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:04.281294 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fc2ed-eb64-494d-8529-9c86135f0068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:04.281973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fc2ed-eb64-494d-8529-9c86135f0068 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:04.282261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fc2ed-eb64-494d-8529-9c86135f0068 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:04.282607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fc2ed-eb64-494d-8529-9c86135f0068 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:04.283330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8fc2ed-eb64-494d-8529-9c86135f0068 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:04.285340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8fc2ed-eb64-494d-8529-9c86135f0068 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:04.340790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d93b0-9cd1-4cf3-9388-78b5e7a1463c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:04.378317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576b88d-552a-4b1b-8fa9-6416a97c7354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:04.379046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576b88d-552a-4b1b-8fa9-6416a97c7354 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:04.379339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576b88d-552a-4b1b-8fa9-6416a97c7354 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:04.379626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576b88d-552a-4b1b-8fa9-6416a97c7354 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:04.380399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576b88d-552a-4b1b-8fa9-6416a97c7354 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:04.384343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576b88d-552a-4b1b-8fa9-6416a97c7354 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:04.399381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 19/37] 10.222.0.45 () {60 vars in 1131 bytes} [Tue May 24 12:45:04 2022] POST /identity/v3/roles => generated 225 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:04.400006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576b88d-552a-4b1b-8fa9-6416a97c7354 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:04.447407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 19/38] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:45:04 2022] POST /identity/v3/projects => generated 301 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:04.448991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3b914-5d9e-4578-9f3c-b55bbfbe106f None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:04.449288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3b914-5d9e-4578-9f3c-b55bbfbe106f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:04.449577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3b914-5d9e-4578-9f3c-b55bbfbe106f None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:04.450996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 20/39] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:04.457223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0c38c-49e4-4f24-91c1-c79c8cacc8e2 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:04.457764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0c38c-49e4-4f24-91c1-c79c8cacc8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:04.458179 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0c38c-49e4-4f24-91c1-c79c8cacc8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:04.481832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9fc0c38c-49e4-4f24-91c1-c79c8cacc8e2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:04.484701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9fc0c38c-49e4-4f24-91c1-c79c8cacc8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:04.537290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:04.557131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:04.570716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576b88d-552a-4b1b-8fa9-6416a97c7354 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:04.618139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 21/40] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:04.619791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677559f-6343-4566-a31a-8de13c06b220 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:04.619897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980151b-c5ed-4ddf-93bf-b7420fc2f256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:04.620100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677559f-6343-4566-a31a-8de13c06b220 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:04.620364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677559f-6343-4566-a31a-8de13c06b220 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:04.620679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980151b-c5ed-4ddf-93bf-b7420fc2f256 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:04.620968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980151b-c5ed-4ddf-93bf-b7420fc2f256 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:04.621252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980151b-c5ed-4ddf-93bf-b7420fc2f256 None admin] PATH_INFO: `/v3/domains/default/users/c0168c20cbe64bb7a4d17219c4fe3eef/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:04.621762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 22/41] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:04.632331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980151b-c5ed-4ddf-93bf-b7420fc2f256 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c0168c20cbe64bb7a4d17219c4fe3eef, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:04.633862 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980151b-c5ed-4ddf-93bf-b7420fc2f256 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:04.635368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5936c-5aeb-478c-83e4-f87d47a785df None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:04.635659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5936c-5aeb-478c-83e4-f87d47a785df None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:04.635938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5936c-5aeb-478c-83e4-f87d47a785df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:04.659981 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 20/42] 10.222.0.45 () {58 vars in 1361 bytes} [Tue May 24 12:45:04 2022] PUT /identity/v3/domains/default/users/c0168c20cbe64bb7a4d17219c4fe3eef/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:04.660977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980151b-c5ed-4ddf-93bf-b7420fc2f256 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:04.662372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-23a5936c-5aeb-478c-83e4-f87d47a785df None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:04.663613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-23a5936c-5aeb-478c-83e4-f87d47a785df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:04.756582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 23/43] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:04.780689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5936c-5aeb-478c-83e4-f87d47a785df None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:04.804273 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980151b-c5ed-4ddf-93bf-b7420fc2f256 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:04.845632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf179b-ee0c-43e8-8eb4-2296ef22c999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:04.846613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf179b-ee0c-43e8-8eb4-2296ef22c999 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:04.846862 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf179b-ee0c-43e8-8eb4-2296ef22c999 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:04.847069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf179b-ee0c-43e8-8eb4-2296ef22c999 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:04.848294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf179b-ee0c-43e8-8eb4-2296ef22c999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:04.849772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf179b-ee0c-43e8-8eb4-2296ef22c999 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:04.851703 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 21/44] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:04.857444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf179b-ee0c-43e8-8eb4-2296ef22c999 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:04.873939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:04.905755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:04.911162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5936c-5aeb-478c-83e4-f87d47a785df None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:04.947427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424985c6-b21b-4a3c-8bb0-ed2ae5b31c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:04.948119 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424985c6-b21b-4a3c-8bb0-ed2ae5b31c94 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:04.948411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424985c6-b21b-4a3c-8bb0-ed2ae5b31c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:04.948698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424985c6-b21b-4a3c-8bb0-ed2ae5b31c94 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:04.949456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424985c6-b21b-4a3c-8bb0-ed2ae5b31c94 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:04.951018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424985c6-b21b-4a3c-8bb0-ed2ae5b31c94 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:04.965629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 24/45] 10.222.0.45 () {60 vars in 1131 bytes} [Tue May 24 12:45:04 2022] POST /identity/v3/roles => generated 223 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:45:04.972074 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf179b-ee0c-43e8-8eb4-2296ef22c999 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:05.010426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d1b67-13c6-44d2-be8a-5dad763921ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:05.011120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1b67-13c6-44d2-be8a-5dad763921ae None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:05.011414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1b67-13c6-44d2-be8a-5dad763921ae None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:05.011699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1b67-13c6-44d2-be8a-5dad763921ae None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:05.012744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d1b67-13c6-44d2-be8a-5dad763921ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:05.014209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d1b67-13c6-44d2-be8a-5dad763921ae None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:05.050339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 22/46] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:45:04 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:45:06.909787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91f6b0-98b5-4204-87cd-076500422c9e None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:06.910328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91f6b0-98b5-4204-87cd-076500422c9e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:06.910438 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91f6b0-98b5-4204-87cd-076500422c9e None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:06.911766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 25/47] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:06.917093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968d4a5-9bcb-4056-a00d-1583fd0878f5 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:06.917465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968d4a5-9bcb-4056-a00d-1583fd0878f5 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:06.917666 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968d4a5-9bcb-4056-a00d-1583fd0878f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:06.935361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-4968d4a5-9bcb-4056-a00d-1583fd0878f5 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:06.936530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4968d4a5-9bcb-4056-a00d-1583fd0878f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:06.992912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:07.002527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 23/48] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:07.025331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91f6b0-98b5-4204-87cd-076500422c9e None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:07.128972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91f6b0-98b5-4204-87cd-076500422c9e None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:07.163677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472c12e-dc9e-4fa9-94fd-c4861e0bb1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:07.164406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472c12e-dc9e-4fa9-94fd-c4861e0bb1fb None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:07.164697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472c12e-dc9e-4fa9-94fd-c4861e0bb1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:07.164978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472c12e-dc9e-4fa9-94fd-c4861e0bb1fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:07.175510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472c12e-dc9e-4fa9-94fd-c4861e0bb1fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-e472c12e-dc9e-4fa9-94fd-c4861e0bb1fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:07.183083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:07.184037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:07.184863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 26/49] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:45:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:07.190243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968d4a5-9bcb-4056-a00d-1583fd0878f5 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:07.260497 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc0325-7765-4b08-8e7e-f76fc83e985a None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:07.260796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc0325-7765-4b08-8e7e-f76fc83e985a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:07.261088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc0325-7765-4b08-8e7e-f76fc83e985a None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:07.263336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 27/50] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:45:07.268925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95bbd9-67f6-43ab-a6c0-3a2b5242c1dd None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:07.269217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95bbd9-67f6-43ab-a6c0-3a2b5242c1dd None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:07.269503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95bbd9-67f6-43ab-a6c0-3a2b5242c1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:07.285575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-5b95bbd9-67f6-43ab-a6c0-3a2b5242c1dd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:07.286809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5b95bbd9-67f6-43ab-a6c0-3a2b5242c1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:07.287558 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968d4a5-9bcb-4056-a00d-1583fd0878f5 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:07.299747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:07.321678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327268a-7eb2-434a-ad8c-9b5fc872481e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:07.322377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327268a-7eb2-434a-ad8c-9b5fc872481e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:07.322665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327268a-7eb2-434a-ad8c-9b5fc872481e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:07.322944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327268a-7eb2-434a-ad8c-9b5fc872481e None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:07.323701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327268a-7eb2-434a-ad8c-9b5fc872481e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:07.324011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327268a-7eb2-434a-ad8c-9b5fc872481e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:07.325393 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327268a-7eb2-434a-ad8c-9b5fc872481e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:07.337385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 24/51] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:07.342819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327268a-7eb2-434a-ad8c-9b5fc872481e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:07.353649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 28/52] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:07.376034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95bbd9-67f6-43ab-a6c0-3a2b5242c1dd None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:07.399896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:07.452099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327268a-7eb2-434a-ad8c-9b5fc872481e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:07.486254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417f819-1d54-4ff4-9981-5a93f794301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:07.487177 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417f819-1d54-4ff4-9981-5a93f794301b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:07.487465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417f819-1d54-4ff4-9981-5a93f794301b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:07.487742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417f819-1d54-4ff4-9981-5a93f794301b None admin] PATH_INFO: `/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:07.488953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417f819-1d54-4ff4-9981-5a93f794301b None admin] RBAC: Authorizing `identity:get_user(user_id=c0168c20cbe64bb7a4d17219c4fe3eef)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:07.490438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417f819-1d54-4ff4-9981-5a93f794301b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:07.490541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95bbd9-67f6-43ab-a6c0-3a2b5242c1dd None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:07.492044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 25/53] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:07 2022] GET /identity/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:07.497245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417f819-1d54-4ff4-9981-5a93f794301b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:07.524626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832536d3-6840-41de-8066-b10e3a51d045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:07.525433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832536d3-6840-41de-8066-b10e3a51d045 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:07.525726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832536d3-6840-41de-8066-b10e3a51d045 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:07.526010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832536d3-6840-41de-8066-b10e3a51d045 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:07.527230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832536d3-6840-41de-8066-b10e3a51d045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:07.528755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832536d3-6840-41de-8066-b10e3a51d045 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:07.530334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 29/54] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:07.536435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832536d3-6840-41de-8066-b10e3a51d045 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:07.553537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:07.599529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417f819-1d54-4ff4-9981-5a93f794301b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:07.621474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:07.632425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b14fc2-753d-4c9e-88f0-47cbb143c9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:07.633228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b14fc2-753d-4c9e-88f0-47cbb143c9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:07.633519 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b14fc2-753d-4c9e-88f0-47cbb143c9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:07.633774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832536d3-6840-41de-8066-b10e3a51d045 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:07.633875 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b14fc2-753d-4c9e-88f0-47cbb143c9f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:07.634982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b14fc2-753d-4c9e-88f0-47cbb143c9f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:07.636442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b14fc2-753d-4c9e-88f0-47cbb143c9f6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:07.638306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 26/55] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:07.644305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b14fc2-753d-4c9e-88f0-47cbb143c9f6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:07.668124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a22b1-4296-497f-b0dc-4859001899ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:07.668784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a22b1-4296-497f-b0dc-4859001899ff None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:07.669071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a22b1-4296-497f-b0dc-4859001899ff None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:07.669356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a22b1-4296-497f-b0dc-4859001899ff None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:07.670065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a22b1-4296-497f-b0dc-4859001899ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:07.671574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a22b1-4296-497f-b0dc-4859001899ff None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:07.698214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 30/56] 10.222.0.45 () {60 vars in 1140 bytes} [Tue May 24 12:45:07 2022] POST /identity/v3/projects => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:07.767014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b14fc2-753d-4c9e-88f0-47cbb143c9f6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:07.802499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e14b37-7aa2-4e7d-aee6-778caf4b2031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:07.803189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e14b37-7aa2-4e7d-aee6-778caf4b2031 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:07.803480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e14b37-7aa2-4e7d-aee6-778caf4b2031 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:07.803764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e14b37-7aa2-4e7d-aee6-778caf4b2031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:07.804638 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e14b37-7aa2-4e7d-aee6-778caf4b2031 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.domain.id=default, user.id=c0168c20cbe64bb7a4d17219c4fe3eef) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:07.804961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e14b37-7aa2-4e7d-aee6-778caf4b2031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:07.806467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e14b37-7aa2-4e7d-aee6-778caf4b2031 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:07.815773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 27/57] 10.222.0.45 () {56 vars in 1322 bytes} [Tue May 24 12:45:07 2022] GET /identity/v3/role_assignments?user.id=c0168c20cbe64bb7a4d17219c4fe3eef&scope.domain.id=default&role.id=0c473678b281495f86f58b077be04e1b => generated 530 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:09.882867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa86c2-d922-43c2-abcb-658cbabd6180 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:09.883367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa86c2-d922-43c2-abcb-658cbabd6180 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:09.883471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa86c2-d922-43c2-abcb-658cbabd6180 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:09.884784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 31/58] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:09.890201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcedd27-b87b-4a2c-9071-c4b1b5aee44e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:09.890640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcedd27-b87b-4a2c-9071-c4b1b5aee44e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:09.890782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcedd27-b87b-4a2c-9071-c4b1b5aee44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:09.907845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-fdcedd27-b87b-4a2c-9071-c4b1b5aee44e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:09.909616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-fdcedd27-b87b-4a2c-9071-c4b1b5aee44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:09.957932 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:09.973467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 28/59] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:09.996898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa86c2-d922-43c2-abcb-658cbabd6180 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:10.063067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:10.097875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa86c2-d922-43c2-abcb-658cbabd6180 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:10.129844 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e350e-8932-444a-ae02-846ee8281e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:10.130666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e350e-8932-444a-ae02-846ee8281e2b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:10.130956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e350e-8932-444a-ae02-846ee8281e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:10.131237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e350e-8932-444a-ae02-846ee8281e2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:10.132413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e350e-8932-444a-ae02-846ee8281e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:10.133926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e350e-8932-444a-ae02-846ee8281e2b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:10.135549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 32/60] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:10.141295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcedd27-b87b-4a2c-9071-c4b1b5aee44e None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:10.238539 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcedd27-b87b-4a2c-9071-c4b1b5aee44e None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:10.271533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9363b93-b8f0-4988-9b8e-202fc96bf128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:10.272196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9363b93-b8f0-4988-9b8e-202fc96bf128 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:10.272485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9363b93-b8f0-4988-9b8e-202fc96bf128 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:10.272766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9363b93-b8f0-4988-9b8e-202fc96bf128 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:10.273484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9363b93-b8f0-4988-9b8e-202fc96bf128 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:10.274961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9363b93-b8f0-4988-9b8e-202fc96bf128 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:10.299300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 29/61] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:45:10 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:13.338907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53ee24-a70e-49bf-af42-27a99f58eb81 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:13.339415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53ee24-a70e-49bf-af42-27a99f58eb81 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:13.339543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53ee24-a70e-49bf-af42-27a99f58eb81 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:13.341302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 33/62] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:13.346979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985759d3-8a38-44f9-938d-afae213bde38 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:13.347722 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985759d3-8a38-44f9-938d-afae213bde38 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:13.348171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985759d3-8a38-44f9-938d-afae213bde38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:13.375269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-985759d3-8a38-44f9-938d-afae213bde38 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:13.376672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-985759d3-8a38-44f9-938d-afae213bde38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:13.429680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:13.499095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 30/63] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:13.534533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53ee24-a70e-49bf-af42-27a99f58eb81 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:13.684396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53ee24-a70e-49bf-af42-27a99f58eb81 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:13.725886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44603a-aba3-40fa-b0c8-bbab4519b093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:13.726730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44603a-aba3-40fa-b0c8-bbab4519b093 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:13.727027 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44603a-aba3-40fa-b0c8-bbab4519b093 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:13.727307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44603a-aba3-40fa-b0c8-bbab4519b093 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:13.730260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:13.743132 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44603a-aba3-40fa-b0c8-bbab4519b093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-8b44603a-aba3-40fa-b0c8-bbab4519b093 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:13.750934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:13.751931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:13.753538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 34/64] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:13.760220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985759d3-8a38-44f9-938d-afae213bde38 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:13.870642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa246a-ef2b-4a34-ae85-3f79b9e0bd1a None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:13.871280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa246a-ef2b-4a34-ae85-3f79b9e0bd1a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:13.871766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa246a-ef2b-4a34-ae85-3f79b9e0bd1a None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:13.873367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 35/65] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:13.879321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da84aa-e5de-4430-ae79-673e9b2326db None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:13.879851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da84aa-e5de-4430-ae79-673e9b2326db None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:13.880343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da84aa-e5de-4430-ae79-673e9b2326db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:13.914338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-42da84aa-e5de-4430-ae79-673e9b2326db None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:13.916223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-42da84aa-e5de-4430-ae79-673e9b2326db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:13.943774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985759d3-8a38-44f9-938d-afae213bde38 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:13.978987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cdc07-c23d-4d5c-9530-bb6392cd3c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:13.979948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cdc07-c23d-4d5c-9530-bb6392cd3c68 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:13.980407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cdc07-c23d-4d5c-9530-bb6392cd3c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:13.980863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cdc07-c23d-4d5c-9530-bb6392cd3c68 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:13.981811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cdc07-c23d-4d5c-9530-bb6392cd3c68 None admin] RBAC: Adding query filter params (name=member) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:13.982307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cdc07-c23d-4d5c-9530-bb6392cd3c68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:13.983912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cdc07-c23d-4d5c-9530-bb6392cd3c68 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:13.986998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 36/66] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:13.988565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f375b-13e0-4a1b-95d7-e6ef125a24c0 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:13.989080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f375b-13e0-4a1b-95d7-e6ef125a24c0 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:13.989558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f375b-13e0-4a1b-95d7-e6ef125a24c0 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:13.991061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 37/67] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:13.998314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aec0d3-2b2a-4c94-a88d-799cea80efe8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:13.998314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aec0d3-2b2a-4c94-a88d-799cea80efe8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:13.998314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aec0d3-2b2a-4c94-a88d-799cea80efe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.001901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 31/68] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:14.011241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cdc07-c23d-4d5c-9530-bb6392cd3c68 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:14.020755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-53aec0d3-2b2a-4c94-a88d-799cea80efe8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:14.022115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-53aec0d3-2b2a-4c94-a88d-799cea80efe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:14.050615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:14.114029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 38/69] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:14.115324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aec0d3-2b2a-4c94-a88d-799cea80efe8 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:14.150611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cdc07-c23d-4d5c-9530-bb6392cd3c68 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:14.176430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:14.194924 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aabdc7-04c8-4936-bf44-5a0531907c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:14.196862 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aabdc7-04c8-4936-bf44-5a0531907c90 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.197373 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aabdc7-04c8-4936-bf44-5a0531907c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.197850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aabdc7-04c8-4936-bf44-5a0531907c90 None admin] PATH_INFO: `/v3/users/944f4c75142549088d2481820e193ef8` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.214881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aabdc7-04c8-4936-bf44-5a0531907c90 None admin] RBAC: Authorizing `identity:get_user(user_id=944f4c75142549088d2481820e193ef8)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:14.216643 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aabdc7-04c8-4936-bf44-5a0531907c90 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:14.218587 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 32/70] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:14 2022] GET /identity/v3/users/944f4c75142549088d2481820e193ef8 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:14.219646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aabdc7-04c8-4936-bf44-5a0531907c90 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:14.243974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aec0d3-2b2a-4c94-a88d-799cea80efe8 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:14.291417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b99023-65f5-48ff-98ab-e303b9fa46a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:14.292485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b99023-65f5-48ff-98ab-e303b9fa46a8 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.293353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b99023-65f5-48ff-98ab-e303b9fa46a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.293641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b99023-65f5-48ff-98ab-e303b9fa46a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.305762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b99023-65f5-48ff-98ab-e303b9fa46a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-01b99023-65f5-48ff-98ab-e303b9fa46a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:14.313820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:14.315055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 39/71] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:45:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:14.316004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b99023-65f5-48ff-98ab-e303b9fa46a8 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:14.321492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:14.331215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aabdc7-04c8-4936-bf44-5a0531907c90 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:14.365263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12864e58-f577-4d8d-ab75-35d9f0d729bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:14.366067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12864e58-f577-4d8d-ab75-35d9f0d729bf None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.366392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12864e58-f577-4d8d-ab75-35d9f0d729bf None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.366678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12864e58-f577-4d8d-ab75-35d9f0d729bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.376693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12864e58-f577-4d8d-ab75-35d9f0d729bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-12864e58-f577-4d8d-ab75-35d9f0d729bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:14.384456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:14.385517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:14.386454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 33/72] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:45:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:14.391603 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f3105-ecea-4370-81c3-f6b99fba4bc3 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.391905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f3105-ecea-4370-81c3-f6b99fba4bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.392194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f3105-ecea-4370-81c3-f6b99fba4bc3 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.394252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 34/73] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:14.395253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493776ce-77f3-46a3-993e-47c2565eb970 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.395546 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493776ce-77f3-46a3-993e-47c2565eb970 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.395955 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493776ce-77f3-46a3-993e-47c2565eb970 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.397181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 35/74] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:14.397930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493776ce-77f3-46a3-993e-47c2565eb970 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:14.426229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b99023-65f5-48ff-98ab-e303b9fa46a8 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:14.442686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:14.460837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f048c-3860-4982-b867-0ed618bb7da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:14.461426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f048c-3860-4982-b867-0ed618bb7da7 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.461709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f048c-3860-4982-b867-0ed618bb7da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.461992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f048c-3860-4982-b867-0ed618bb7da7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.499162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493776ce-77f3-46a3-993e-47c2565eb970 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:14.529457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f048c-3860-4982-b867-0ed618bb7da7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:14.531089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f048c-3860-4982-b867-0ed618bb7da7 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:14.532557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f582a-81f7-4334-bcd1-ad6aeb92bad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:14.533229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f582a-81f7-4334-bcd1-ad6aeb92bad9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.534002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f582a-81f7-4334-bcd1-ad6aeb92bad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.534306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f582a-81f7-4334-bcd1-ad6aeb92bad9 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.535073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f582a-81f7-4334-bcd1-ad6aeb92bad9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:14.535389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f582a-81f7-4334-bcd1-ad6aeb92bad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:14.536753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f582a-81f7-4334-bcd1-ad6aeb92bad9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:14.544564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 36/75] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:45:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:14.549852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f582a-81f7-4334-bcd1-ad6aeb92bad9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:14.565266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:14.630872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 40/76] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:14.632472 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bcc66-dd19-43c4-8df1-9f23352d08b8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.632773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bcc66-dd19-43c4-8df1-9f23352d08b8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.633103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bcc66-dd19-43c4-8df1-9f23352d08b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.648497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f582a-81f7-4334-bcd1-ad6aeb92bad9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:14.648936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-bf7bcc66-dd19-43c4-8df1-9f23352d08b8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:14.650115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bf7bcc66-dd19-43c4-8df1-9f23352d08b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:14.660973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:14.682423 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6999ce-390a-48a6-a4ce-41a7ad73781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:14.683100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6999ce-390a-48a6-a4ce-41a7ad73781e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.683391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6999ce-390a-48a6-a4ce-41a7ad73781e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.683674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6999ce-390a-48a6-a4ce-41a7ad73781e None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.684429 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6999ce-390a-48a6-a4ce-41a7ad73781e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:14.684744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6999ce-390a-48a6-a4ce-41a7ad73781e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:14.686145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6999ce-390a-48a6-a4ce-41a7ad73781e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:14.695196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 37/77] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:14.696183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67bf5-4035-43e9-a000-2822d57424e8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.696475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67bf5-4035-43e9-a000-2822d57424e8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.696764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67bf5-4035-43e9-a000-2822d57424e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.713450 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-bba67bf5-4035-43e9-a000-2822d57424e8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:14.714670 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-bba67bf5-4035-43e9-a000-2822d57424e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:14.721983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 41/78] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:14.723407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bcc66-dd19-43c4-8df1-9f23352d08b8 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:14.781874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 38/79] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:14.782743 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba67bf5-4035-43e9-a000-2822d57424e8 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:14.790714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:14.831519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bcc66-dd19-43c4-8df1-9f23352d08b8 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:14.865251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423ccc0-bbb2-4a59-991b-b862e0235c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:14.866123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423ccc0-bbb2-4a59-991b-b862e0235c76 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.866449 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423ccc0-bbb2-4a59-991b-b862e0235c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.866733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423ccc0-bbb2-4a59-991b-b862e0235c76 None admin] PATH_INFO: `/v3/users/944f4c75142549088d2481820e193ef8` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.867933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423ccc0-bbb2-4a59-991b-b862e0235c76 None admin] RBAC: Authorizing `identity:get_user(user_id=944f4c75142549088d2481820e193ef8)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:14.869377 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423ccc0-bbb2-4a59-991b-b862e0235c76 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:14.871211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 42/80] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:14 2022] GET /identity/v3/users/944f4c75142549088d2481820e193ef8 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:14.871793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423ccc0-bbb2-4a59-991b-b862e0235c76 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:14.888637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:14.893801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba67bf5-4035-43e9-a000-2822d57424e8 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:14.927267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db829fa-a4e4-4b52-91c9-b7bd1a3715bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:14.928133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db829fa-a4e4-4b52-91c9-b7bd1a3715bb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:14.928424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db829fa-a4e4-4b52-91c9-b7bd1a3715bb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:14.928713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db829fa-a4e4-4b52-91c9-b7bd1a3715bb None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:14.929911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db829fa-a4e4-4b52-91c9-b7bd1a3715bb None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:14.931397 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db829fa-a4e4-4b52-91c9-b7bd1a3715bb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:14.933078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 39/81] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:14 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:14.933834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db829fa-a4e4-4b52-91c9-b7bd1a3715bb None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:14.972421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423ccc0-bbb2-4a59-991b-b862e0235c76 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:15.006213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761ef52-b664-49b5-9eab-8147de0c2960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:15.007211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761ef52-b664-49b5-9eab-8147de0c2960 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:15.007598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761ef52-b664-49b5-9eab-8147de0c2960 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:15.007889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761ef52-b664-49b5-9eab-8147de0c2960 None admin] PATH_INFO: `/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:15.009087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761ef52-b664-49b5-9eab-8147de0c2960 None admin] RBAC: Authorizing `identity:get_user(user_id=c0168c20cbe64bb7a4d17219c4fe3eef)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:15.010387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:15.010590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761ef52-b664-49b5-9eab-8147de0c2960 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:15.012805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 43/82] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:14 2022] GET /identity/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:15.013390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761ef52-b664-49b5-9eab-8147de0c2960 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:15.035848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db829fa-a4e4-4b52-91c9-b7bd1a3715bb None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:15.069086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d177e2a-c3a6-48a4-81c6-b4de3cfe7057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:15.069892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d177e2a-c3a6-48a4-81c6-b4de3cfe7057 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:15.070182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d177e2a-c3a6-48a4-81c6-b4de3cfe7057 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:15.070501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d177e2a-c3a6-48a4-81c6-b4de3cfe7057 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:15.071734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d177e2a-c3a6-48a4-81c6-b4de3cfe7057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:15.073154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d177e2a-c3a6-48a4-81c6-b4de3cfe7057 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:15.074812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 40/83] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:15.075575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d177e2a-c3a6-48a4-81c6-b4de3cfe7057 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:15.093799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:15.112892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761ef52-b664-49b5-9eab-8147de0c2960 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:15.153838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb44e8-db71-4336-8e96-5e2e8da15da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:15.154717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb44e8-db71-4336-8e96-5e2e8da15da9 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:15.154939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb44e8-db71-4336-8e96-5e2e8da15da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:15.155215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb44e8-db71-4336-8e96-5e2e8da15da9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:15.166046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb44e8-db71-4336-8e96-5e2e8da15da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-38eb44e8-db71-4336-8e96-5e2e8da15da9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:15.173741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:15.175156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:15.181184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:15.181908 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d177e2a-c3a6-48a4-81c6-b4de3cfe7057 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:15.181988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 44/84] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:15.181988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb44e8-db71-4336-8e96-5e2e8da15da9 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:15.208738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cd34b-4e75-48c1-b708-e3eda9701813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:15.209428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cd34b-4e75-48c1-b708-e3eda9701813 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:15.209716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cd34b-4e75-48c1-b708-e3eda9701813 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:15.209996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cd34b-4e75-48c1-b708-e3eda9701813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:15.222635 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:15.276788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cd34b-4e75-48c1-b708-e3eda9701813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:15.277863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb44e8-db71-4336-8e96-5e2e8da15da9 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:15.278269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cd34b-4e75-48c1-b708-e3eda9701813 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:15.311536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383812d-22d1-47f3-aac3-20e1f7597483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:15.312297 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383812d-22d1-47f3-aac3-20e1f7597483 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:15.312594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383812d-22d1-47f3-aac3-20e1f7597483 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:15.312876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383812d-22d1-47f3-aac3-20e1f7597483 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:15.313716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383812d-22d1-47f3-aac3-20e1f7597483 None admin] RBAC: Adding query filter params (role.id=27d8d0ea0b83488a95a9141d439247f6, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162, user.id=944f4c75142549088d2481820e193ef8) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:15.314037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383812d-22d1-47f3-aac3-20e1f7597483 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:15.315556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383812d-22d1-47f3-aac3-20e1f7597483 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:15.320911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 45/85] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:15 2022] GET /identity/v3/role_assignments?user.id=944f4c75142549088d2481820e193ef8&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:15.321700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383812d-22d1-47f3-aac3-20e1f7597483 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:15.341911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:15.381163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 41/86] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:15.387538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cd34b-4e75-48c1-b708-e3eda9701813 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:15.435161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:15.462124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383812d-22d1-47f3-aac3-20e1f7597483 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:15.499519 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cd34b-4e75-48c1-b708-e3eda9701813 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:15.500967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8635580-6f2f-4c65-a51f-fd004bc1596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:15.501716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635580-6f2f-4c65-a51f-fd004bc1596f None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:15.502039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635580-6f2f-4c65-a51f-fd004bc1596f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:15.502420 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635580-6f2f-4c65-a51f-fd004bc1596f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:15.544069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031e3b0-fb97-4671-8369-3c42fea7d259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:15.545302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031e3b0-fb97-4671-8369-3c42fea7d259 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:15.545597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031e3b0-fb97-4671-8369-3c42fea7d259 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:15.545924 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031e3b0-fb97-4671-8369-3c42fea7d259 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:15.547018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031e3b0-fb97-4671-8369-3c42fea7d259 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:15.548469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031e3b0-fb97-4671-8369-3c42fea7d259 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:15.578316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 42/87] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:45:15 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:15.578316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031e3b0-fb97-4671-8369-3c42fea7d259 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:15.587254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8635580-6f2f-4c65-a51f-fd004bc1596f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:15.589215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8635580-6f2f-4c65-a51f-fd004bc1596f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:15.602070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:15.709036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:15.713073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 46/88] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:15.714517 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8635580-6f2f-4c65-a51f-fd004bc1596f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:15.719601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031e3b0-fb97-4671-8369-3c42fea7d259 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:15.769097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eba646-ad83-459c-9c40-f92e547db3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:15.769097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eba646-ad83-459c-9c40-f92e547db3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:15.769328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eba646-ad83-459c-9c40-f92e547db3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:15.769620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eba646-ad83-459c-9c40-f92e547db3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:15.772007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eba646-ad83-459c-9c40-f92e547db3f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:15.772007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eba646-ad83-459c-9c40-f92e547db3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:15.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eba646-ad83-459c-9c40-f92e547db3f6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:15.784714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 43/89] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:15.785560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eba646-ad83-459c-9c40-f92e547db3f6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:15.833368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8635580-6f2f-4c65-a51f-fd004bc1596f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:15.841221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:15.878801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e301dd6-95e1-4535-988a-bf82c02652c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:15.879677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301dd6-95e1-4535-988a-bf82c02652c9 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:15.879962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301dd6-95e1-4535-988a-bf82c02652c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:15.880239 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301dd6-95e1-4535-988a-bf82c02652c9 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:15.881523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e301dd6-95e1-4535-988a-bf82c02652c9 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:15.883028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e301dd6-95e1-4535-988a-bf82c02652c9 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:15.885017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 47/90] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:15 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:15.886801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e301dd6-95e1-4535-988a-bf82c02652c9 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:15.938606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eba646-ad83-459c-9c40-f92e547db3f6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:15.984046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac2f41-ffb5-4e64-bdb3-779037dc5a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:15.985524 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac2f41-ffb5-4e64-bdb3-779037dc5a19 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:15.985989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac2f41-ffb5-4e64-bdb3-779037dc5a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:15.986728 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac2f41-ffb5-4e64-bdb3-779037dc5a19 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:15.988998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac2f41-ffb5-4e64-bdb3-779037dc5a19 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:15.990666 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac2f41-ffb5-4e64-bdb3-779037dc5a19 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:15.993183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 44/91] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:15 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:15.994125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac2f41-ffb5-4e64-bdb3-779037dc5a19 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:16.025752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e301dd6-95e1-4535-988a-bf82c02652c9 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:16.034502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:16.065212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4182e-64ac-45e7-b10c-bb1baadc5856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:16.066428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4182e-64ac-45e7-b10c-bb1baadc5856 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:16.066894 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4182e-64ac-45e7-b10c-bb1baadc5856 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:16.067430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4182e-64ac-45e7-b10c-bb1baadc5856 None admin] PATH_INFO: `/v3/users/944f4c75142549088d2481820e193ef8` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:16.069024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4182e-64ac-45e7-b10c-bb1baadc5856 None admin] RBAC: Authorizing `identity:get_user(user_id=944f4c75142549088d2481820e193ef8)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:16.070873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4182e-64ac-45e7-b10c-bb1baadc5856 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:16.073097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 48/92] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:15 2022] GET /identity/v3/users/944f4c75142549088d2481820e193ef8 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:16.074191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4182e-64ac-45e7-b10c-bb1baadc5856 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:16.118965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:16.129021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac2f41-ffb5-4e64-bdb3-779037dc5a19 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:16.179496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4182e-64ac-45e7-b10c-bb1baadc5856 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:16.208457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa023fa-e274-4e9b-8b8b-59d8b4f4ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:16.209130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa023fa-e274-4e9b-8b8b-59d8b4f4ba3c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:16.209399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa023fa-e274-4e9b-8b8b-59d8b4f4ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:16.209679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa023fa-e274-4e9b-8b8b-59d8b4f4ba3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:16.210553 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa023fa-e274-4e9b-8b8b-59d8b4f4ba3c None admin] RBAC: Adding query filter params (role.id=c7b7d3b4dd944936851dee41ead5dc99, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162, user.id=944f4c75142549088d2481820e193ef8) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:16.211254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa023fa-e274-4e9b-8b8b-59d8b4f4ba3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:16.213544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa023fa-e274-4e9b-8b8b-59d8b4f4ba3c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:16.223904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb885d-6277-4767-95ed-5eff2f266a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:16.224909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb885d-6277-4767-95ed-5eff2f266a18 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:16.225392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb885d-6277-4767-95ed-5eff2f266a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:16.225909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb885d-6277-4767-95ed-5eff2f266a18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:16.227068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb885d-6277-4767-95ed-5eff2f266a18 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162, user.id=c0168c20cbe64bb7a4d17219c4fe3eef) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:16.227753 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb885d-6277-4767-95ed-5eff2f266a18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:16.229489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb885d-6277-4767-95ed-5eff2f266a18 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:16.231579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 45/93] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:15 2022] GET /identity/v3/role_assignments?user.id=944f4c75142549088d2481820e193ef8&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=c7b7d3b4dd944936851dee41ead5dc99 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:16.232677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa023fa-e274-4e9b-8b8b-59d8b4f4ba3c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:16.242634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 49/94] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:16 2022] GET /identity/v3/role_assignments?user.id=c0168c20cbe64bb7a4d17219c4fe3eef&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=0c473678b281495f86f58b077be04e1b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:16.353972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:16.368124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa023fa-e274-4e9b-8b8b-59d8b4f4ba3c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:16.448984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523786e-17f6-44e1-b1ec-d23ddbcc3ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:16.449760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523786e-17f6-44e1-b1ec-d23ddbcc3ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:16.450025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523786e-17f6-44e1-b1ec-d23ddbcc3ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:16.450323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523786e-17f6-44e1-b1ec-d23ddbcc3ac0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:16.541929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523786e-17f6-44e1-b1ec-d23ddbcc3ac0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:16.543504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523786e-17f6-44e1-b1ec-d23ddbcc3ac0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:16.658017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:16.670242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 46/95] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:16.676637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb885d-6277-4767-95ed-5eff2f266a18 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:16.834160 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb885d-6277-4767-95ed-5eff2f266a18 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:16.844133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:16.883852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f02da8-e4bd-40c3-a141-7cebfa6e5784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:16.885057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f02da8-e4bd-40c3-a141-7cebfa6e5784 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:16.885560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f02da8-e4bd-40c3-a141-7cebfa6e5784 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:16.886049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f02da8-e4bd-40c3-a141-7cebfa6e5784 None admin] PATH_INFO: `/v3/projects/bafc86bc275846f5861a1b426ebd7c81` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:16.887744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f02da8-e4bd-40c3-a141-7cebfa6e5784 None admin] RBAC: Authorizing `identity:get_project(project_id=bafc86bc275846f5861a1b426ebd7c81)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:16.889400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f02da8-e4bd-40c3-a141-7cebfa6e5784 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:16.891304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 50/96] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:16 2022] GET /identity/v3/projects/bafc86bc275846f5861a1b426ebd7c81 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:16.897222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523786e-17f6-44e1-b1ec-d23ddbcc3ac0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:17.018385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523786e-17f6-44e1-b1ec-d23ddbcc3ac0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:17.057722 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b724e2a-ef22-4b51-8d62-a36ac5b6123d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:17.058759 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b724e2a-ef22-4b51-8d62-a36ac5b6123d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:17.059244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b724e2a-ef22-4b51-8d62-a36ac5b6123d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:17.059739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b724e2a-ef22-4b51-8d62-a36ac5b6123d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:17.060768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b724e2a-ef22-4b51-8d62-a36ac5b6123d None admin] RBAC: Adding query filter params (role.id=27d8d0ea0b83488a95a9141d439247f6, scope.project.id=bafc86bc275846f5861a1b426ebd7c81, user.id=944f4c75142549088d2481820e193ef8) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:17.061276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b724e2a-ef22-4b51-8d62-a36ac5b6123d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:17.062960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b724e2a-ef22-4b51-8d62-a36ac5b6123d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:17.069626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 47/97] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:16 2022] GET /identity/v3/role_assignments?user.id=944f4c75142549088d2481820e193ef8&scope.project.id=bafc86bc275846f5861a1b426ebd7c81&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:18.005714 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab73db-1119-4824-b86f-dd6ec1c26913 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:18.006718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab73db-1119-4824-b86f-dd6ec1c26913 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:18.007206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab73db-1119-4824-b86f-dd6ec1c26913 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:18.008773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 51/98] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:18.027213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3384a-79d1-47cf-94c4-b370c974b4ca None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:18.028179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3384a-79d1-47cf-94c4-b370c974b4ca None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:18.028655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3384a-79d1-47cf-94c4-b370c974b4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:18.051668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-14a3384a-79d1-47cf-94c4-b370c974b4ca None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:18.053181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-14a3384a-79d1-47cf-94c4-b370c974b4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:18.120807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:18.156019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 48/99] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:18.204325 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab73db-1119-4824-b86f-dd6ec1c26913 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:18.338966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab73db-1119-4824-b86f-dd6ec1c26913 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:18.393416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4f3f4-6d84-4ed2-9f38-fea84d01f948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:18.394615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4f3f4-6d84-4ed2-9f38-fea84d01f948 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:18.395116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4f3f4-6d84-4ed2-9f38-fea84d01f948 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:18.395585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4f3f4-6d84-4ed2-9f38-fea84d01f948 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:18.419242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4f3f4-6d84-4ed2-9f38-fea84d01f948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-28d4f3f4-6d84-4ed2-9f38-fea84d01f948 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:18.427485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:18.428320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:18.429071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:18.432537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 52/100] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:18.438552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3384a-79d1-47cf-94c4-b370c974b4ca None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:18.624963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3384a-79d1-47cf-94c4-b370c974b4ca None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:18.693061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd934f-9b4d-40f8-8789-ae8c719b05bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:18.694631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd934f-9b4d-40f8-8789-ae8c719b05bd None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:18.695346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd934f-9b4d-40f8-8789-ae8c719b05bd None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:18.696034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd934f-9b4d-40f8-8789-ae8c719b05bd None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:18.697604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd934f-9b4d-40f8-8789-ae8c719b05bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:18.698434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd934f-9b4d-40f8-8789-ae8c719b05bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:18.701153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd934f-9b4d-40f8-8789-ae8c719b05bd None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:18.722692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 49/101] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:18.731912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4f3f4-6d84-4ed2-9f38-fea84d01f948 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:18.815827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:18.892409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e1e9e-5c63-42f0-aa1f-bb076a2e2668 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:18.893813 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e1e9e-5c63-42f0-aa1f-bb076a2e2668 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:18.894695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e1e9e-5c63-42f0-aa1f-bb076a2e2668 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:18.897424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 50/102] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:18.916321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9e1a2-5a16-412d-8961-b91ab31d38bb None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:18.917166 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9e1a2-5a16-412d-8961-b91ab31d38bb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:18.917912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9e1a2-5a16-412d-8961-b91ab31d38bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:18.964083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-c2f9e1a2-5a16-412d-8961-b91ab31d38bb None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:18.971222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c2f9e1a2-5a16-412d-8961-b91ab31d38bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:19.010315 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4f3f4-6d84-4ed2-9f38-fea84d01f948 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:19.093569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:19.116026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ad80f-284d-43d1-ab44-34202743c3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:19.117208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ad80f-284d-43d1-ab44-34202743c3dc None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:19.117658 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ad80f-284d-43d1-ab44-34202743c3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:19.118076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ad80f-284d-43d1-ab44-34202743c3dc None admin] PATH_INFO: `/v3/users/944f4c75142549088d2481820e193ef8` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:19.119458 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ad80f-284d-43d1-ab44-34202743c3dc None admin] RBAC: Authorizing `identity:get_user(user_id=944f4c75142549088d2481820e193ef8)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:19.121075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ad80f-284d-43d1-ab44-34202743c3dc None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:19.122870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 53/103] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:18 2022] GET /identity/v3/users/944f4c75142549088d2481820e193ef8 => generated 275 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:19.128970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ad80f-284d-43d1-ab44-34202743c3dc None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:19.149826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 51/104] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:19.192001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9e1a2-5a16-412d-8961-b91ab31d38bb None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:19.286232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ad80f-284d-43d1-ab44-34202743c3dc None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:19.288017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:19.333874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d808d60-1ab1-4a70-9bea-81fba3430f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:19.335287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808d60-1ab1-4a70-9bea-81fba3430f90 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:19.335730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808d60-1ab1-4a70-9bea-81fba3430f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:19.336184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808d60-1ab1-4a70-9bea-81fba3430f90 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:19.337709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d808d60-1ab1-4a70-9bea-81fba3430f90 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:19.340023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d808d60-1ab1-4a70-9bea-81fba3430f90 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:19.342650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 54/105] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:19 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:19.348356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8860-3748-435e-9218-600fd5ecdbcf None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:19.348356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8860-3748-435e-9218-600fd5ecdbcf None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:19.348356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8860-3748-435e-9218-600fd5ecdbcf None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:19.348356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 55/106] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:19.349464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcc25c-0807-456c-96b7-9fcc0965abd3 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:19.349567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcc25c-0807-456c-96b7-9fcc0965abd3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:19.351000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcc25c-0807-456c-96b7-9fcc0965abd3 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:19.351119 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 56/107] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:19.351751 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcc25c-0807-456c-96b7-9fcc0965abd3 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:19.371583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9e1a2-5a16-412d-8961-b91ab31d38bb None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:19.411356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31429fa7-d089-415e-877d-5dc077fd1788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:19.412380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31429fa7-d089-415e-877d-5dc077fd1788 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:19.412831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31429fa7-d089-415e-877d-5dc077fd1788 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:19.413253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31429fa7-d089-415e-877d-5dc077fd1788 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:19.425327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31429fa7-d089-415e-877d-5dc077fd1788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-31429fa7-d089-415e-877d-5dc077fd1788 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:19.433501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:19.434601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:19.435561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:19.437630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 52/108] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:19.444032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930f3b5-3f19-4c18-ac68-269056c2710a None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:19.444526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930f3b5-3f19-4c18-ac68-269056c2710a None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:19.444986 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930f3b5-3f19-4c18-ac68-269056c2710a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:19.463344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9930f3b5-3f19-4c18-ac68-269056c2710a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:19.464840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9930f3b5-3f19-4c18-ac68-269056c2710a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:19.475940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcc25c-0807-456c-96b7-9fcc0965abd3 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:19.498017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:19.511411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9698ae-64db-4f73-af9b-baa5d34f0e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:19.512479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9698ae-64db-4f73-af9b-baa5d34f0e59 None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:19.512837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9698ae-64db-4f73-af9b-baa5d34f0e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:19.513188 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9698ae-64db-4f73-af9b-baa5d34f0e59 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/users/944f4c75142549088d2481820e193ef8/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:19.515352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9698ae-64db-4f73-af9b-baa5d34f0e59 None admin] RBAC: Authorizing `identity:create_grant(project_id=a75c6b7e46c24ae8adacf3d5669e7162, user_id=944f4c75142549088d2481820e193ef8, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:19.516889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9698ae-64db-4f73-af9b-baa5d34f0e59 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:19.536180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 57/109] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:45:19 2022] PUT /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/users/944f4c75142549088d2481820e193ef8/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:19.537765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415c09a-d002-4512-b5a6-75dcbc57441b None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:19.538019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415c09a-d002-4512-b5a6-75dcbc57441b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:19.538445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415c09a-d002-4512-b5a6-75dcbc57441b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:19.550455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 53/110] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:19.551759 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930f3b5-3f19-4c18-ac68-269056c2710a None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:19.557101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-5415c09a-d002-4512-b5a6-75dcbc57441b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:19.558569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5415c09a-d002-4512-b5a6-75dcbc57441b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:19.585791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:19.632971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 58/111] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:19.633969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415c09a-d002-4512-b5a6-75dcbc57441b None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:19.682424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930f3b5-3f19-4c18-ac68-269056c2710a None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:19.716965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f755e2-1dec-4d9c-b866-5ffd7305aa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:19.717718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f755e2-1dec-4d9c-b866-5ffd7305aa94 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:19.718011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f755e2-1dec-4d9c-b866-5ffd7305aa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:19.718322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f755e2-1dec-4d9c-b866-5ffd7305aa94 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:19.719133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f755e2-1dec-4d9c-b866-5ffd7305aa94 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:19.719448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f755e2-1dec-4d9c-b866-5ffd7305aa94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:19.721377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f755e2-1dec-4d9c-b866-5ffd7305aa94 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:19.731088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 54/112] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:19.731955 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f755e2-1dec-4d9c-b866-5ffd7305aa94 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:19.769794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:19.787752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415c09a-d002-4512-b5a6-75dcbc57441b None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:19.846882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f755e2-1dec-4d9c-b866-5ffd7305aa94 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:19.857530 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e38da5-bc04-47b8-8582-20c602e1dbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:19.858744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e38da5-bc04-47b8-8582-20c602e1dbab None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:19.859267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e38da5-bc04-47b8-8582-20c602e1dbab None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:19.859762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e38da5-bc04-47b8-8582-20c602e1dbab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:19.861185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e38da5-bc04-47b8-8582-20c602e1dbab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:19.862985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e38da5-bc04-47b8-8582-20c602e1dbab None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:19.874312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 59/113] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:19.875639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e38da5-bc04-47b8-8582-20c602e1dbab None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:19.895339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f40810-3d55-46e0-a079-1f449e2d42b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:19.896913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40810-3d55-46e0-a079-1f449e2d42b1 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:19.897257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40810-3d55-46e0-a079-1f449e2d42b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:19.897542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40810-3d55-46e0-a079-1f449e2d42b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:19.914072 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f40810-3d55-46e0-a079-1f449e2d42b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-f0f40810-3d55-46e0-a079-1f449e2d42b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:19.922050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:19.922926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:19.923725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 55/114] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:45:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:19.924445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f40810-3d55-46e0-a079-1f449e2d42b1 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:19.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:20.023860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:20.035392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f40810-3d55-46e0-a079-1f449e2d42b1 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:20.058218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e38da5-bc04-47b8-8582-20c602e1dbab None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:20.070740 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac58c8-af6a-4c1e-882f-9eb588240278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:20.071414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac58c8-af6a-4c1e-882f-9eb588240278 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:20.071709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac58c8-af6a-4c1e-882f-9eb588240278 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:20.071983 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac58c8-af6a-4c1e-882f-9eb588240278 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:20.072757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac58c8-af6a-4c1e-882f-9eb588240278 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:20.074266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac58c8-af6a-4c1e-882f-9eb588240278 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:20.104688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 56/115] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:45:19 2022] POST /identity/v3/projects => generated 302 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:20.105631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac58c8-af6a-4c1e-882f-9eb588240278 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:20.119672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254c001-6422-4608-a9a3-779537d404d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:20.121174 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254c001-6422-4608-a9a3-779537d404d6 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:20.121668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254c001-6422-4608-a9a3-779537d404d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:20.122153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254c001-6422-4608-a9a3-779537d404d6 None admin] PATH_INFO: `/v3/users/1e91129d27284f81931cb58522ffc41e` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:20.154297 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254c001-6422-4608-a9a3-779537d404d6 None admin] RBAC: Authorizing `identity:get_user(user_id=1e91129d27284f81931cb58522ffc41e)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:20.157413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254c001-6422-4608-a9a3-779537d404d6 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:20.161522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 60/116] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:19 2022] GET /identity/v3/users/1e91129d27284f81931cb58522ffc41e => generated 289 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:20.180745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254c001-6422-4608-a9a3-779537d404d6 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:20.251958 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac58c8-af6a-4c1e-882f-9eb588240278 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:20.288650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a8e8-83b8-4cd0-bfae-a779f371fe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:20.289392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a8e8-83b8-4cd0-bfae-a779f371fe78 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:20.289768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a8e8-83b8-4cd0-bfae-a779f371fe78 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:20.291212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a8e8-83b8-4cd0-bfae-a779f371fe78 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:20.291212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0a8e8-83b8-4cd0-bfae-a779f371fe78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:20.291531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0a8e8-83b8-4cd0-bfae-a779f371fe78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:20.292718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0a8e8-83b8-4cd0-bfae-a779f371fe78 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:20.304523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 57/117] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:20.310223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a8e8-83b8-4cd0-bfae-a779f371fe78 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:20.320965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:20.433424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a8e8-83b8-4cd0-bfae-a779f371fe78 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:20.470337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fac52-656d-43de-adf6-f69b5872e4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:20.471906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fac52-656d-43de-adf6-f69b5872e4ff None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:20.472196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fac52-656d-43de-adf6-f69b5872e4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:20.472477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fac52-656d-43de-adf6-f69b5872e4ff None admin] PATH_INFO: `/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:20.473739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fac52-656d-43de-adf6-f69b5872e4ff None admin] RBAC: Authorizing `identity:get_user(user_id=c0168c20cbe64bb7a4d17219c4fe3eef)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:20.475580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:20.481553 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fac52-656d-43de-adf6-f69b5872e4ff None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:20.484517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 58/118] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:20 2022] GET /identity/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:20.490606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fac52-656d-43de-adf6-f69b5872e4ff None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:20.493656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254c001-6422-4608-a9a3-779537d404d6 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:20.556003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52e98c-90e1-4063-a80b-1aa6b492db1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:20.557029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e98c-90e1-4063-a80b-1aa6b492db1d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:20.557522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e98c-90e1-4063-a80b-1aa6b492db1d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:20.557999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e98c-90e1-4063-a80b-1aa6b492db1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:20.630827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:20.688159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fac52-656d-43de-adf6-f69b5872e4ff None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:20.704383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52e98c-90e1-4063-a80b-1aa6b492db1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:20.706271 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52e98c-90e1-4063-a80b-1aa6b492db1d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:20.750005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7aa9ce-802d-4c06-9f1e-c91e30de4042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:20.751418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7aa9ce-802d-4c06-9f1e-c91e30de4042 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:20.752463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7aa9ce-802d-4c06-9f1e-c91e30de4042 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:20.752945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7aa9ce-802d-4c06-9f1e-c91e30de4042 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:20.754385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7aa9ce-802d-4c06-9f1e-c91e30de4042 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:20.756617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7aa9ce-802d-4c06-9f1e-c91e30de4042 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:20.758692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 59/119] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:20 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:20.766121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7aa9ce-802d-4c06-9f1e-c91e30de4042 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:20.868991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:20.905412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 61/120] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:45:20.909091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52e98c-90e1-4063-a80b-1aa6b492db1d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:20.959317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7aa9ce-802d-4c06-9f1e-c91e30de4042 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:21.045681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748bb44-4d3d-48f5-901a-9496a6f32181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:21.046634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748bb44-4d3d-48f5-901a-9496a6f32181 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:21.046923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748bb44-4d3d-48f5-901a-9496a6f32181 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:21.047207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748bb44-4d3d-48f5-901a-9496a6f32181 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/users/c0168c20cbe64bb7a4d17219c4fe3eef/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:21.049765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748bb44-4d3d-48f5-901a-9496a6f32181 None admin] RBAC: Authorizing `identity:create_grant(project_id=a75c6b7e46c24ae8adacf3d5669e7162, user_id=c0168c20cbe64bb7a4d17219c4fe3eef, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:21.051263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748bb44-4d3d-48f5-901a-9496a6f32181 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:21.056640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52e98c-90e1-4063-a80b-1aa6b492db1d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:21.075201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:21.081750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 60/121] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:45:20 2022] PUT /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/users/c0168c20cbe64bb7a4d17219c4fe3eef/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:21.104866 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aef1b-1e60-4cff-be64-ed30a1ec408a None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:21.105578 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aef1b-1e60-4cff-be64-ed30a1ec408a None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:21.105578 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aef1b-1e60-4cff-be64-ed30a1ec408a None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:21.106677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdd9e9-1f32-404d-b870-3581747363e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:21.107087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 61/122] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:45:21.108031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdd9e9-1f32-404d-b870-3581747363e4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:21.108328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdd9e9-1f32-404d-b870-3581747363e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:21.108616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdd9e9-1f32-404d-b870-3581747363e4 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:21.113559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdd9e9-1f32-404d-b870-3581747363e4 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:21.113755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693af3f5-62ad-47d6-b5f4-6f4ede452159 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:21.113982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693af3f5-62ad-47d6-b5f4-6f4ede452159 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:21.114276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693af3f5-62ad-47d6-b5f4-6f4ede452159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:21.115137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdd9e9-1f32-404d-b870-3581747363e4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:21.116986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 62/123] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:20 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:21.122889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdd9e9-1f32-404d-b870-3581747363e4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:21.133030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-693af3f5-62ad-47d6-b5f4-6f4ede452159 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:21.134226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-693af3f5-62ad-47d6-b5f4-6f4ede452159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:21.262659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:21.269215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 62/124] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:21.308066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693af3f5-62ad-47d6-b5f4-6f4ede452159 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:21.362690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdd9e9-1f32-404d-b870-3581747363e4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:21.445594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f453447-70e8-4163-9a64-1e0029cd4cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:21.446268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f453447-70e8-4163-9a64-1e0029cd4cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:21.446583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f453447-70e8-4163-9a64-1e0029cd4cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:21.446865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f453447-70e8-4163-9a64-1e0029cd4cc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:21.447715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f453447-70e8-4163-9a64-1e0029cd4cc9 None admin] RBAC: Adding query filter params (role.id=4a71e72752894223a2b3fd53f728c211, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162, user.id=1e91129d27284f81931cb58522ffc41e) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:21.448032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f453447-70e8-4163-9a64-1e0029cd4cc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:21.449490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f453447-70e8-4163-9a64-1e0029cd4cc9 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:21.464518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 63/125] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:21 2022] GET /identity/v3/role_assignments?user.id=1e91129d27284f81931cb58522ffc41e&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=4a71e72752894223a2b3fd53f728c211 => generated 259 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:21.508512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693af3f5-62ad-47d6-b5f4-6f4ede452159 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:21.522624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65322644-e34f-4a38-b401-78bf92ae67bd None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:21.522929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65322644-e34f-4a38-b401-78bf92ae67bd None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:21.523197 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65322644-e34f-4a38-b401-78bf92ae67bd None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:21.524601 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 64/126] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:45:21.541219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fed2d-9c52-4c23-bfa0-8e6d769d09e4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:21.541558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fed2d-9c52-4c23-bfa0-8e6d769d09e4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:21.541792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fed2d-9c52-4c23-bfa0-8e6d769d09e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:21.565337 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7a5fed2d-9c52-4c23-bfa0-8e6d769d09e4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:21.567431 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7a5fed2d-9c52-4c23-bfa0-8e6d769d09e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:21.578693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:21.586138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b3831-133d-43c3-bff7-46a1df581ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:21.587022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b3831-133d-43c3-bff7-46a1df581ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:21.587313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b3831-133d-43c3-bff7-46a1df581ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:21.587597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b3831-133d-43c3-bff7-46a1df581ba2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:21.610341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b3831-133d-43c3-bff7-46a1df581ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-8c4b3831-133d-43c3-bff7-46a1df581ba2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:21.620654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:21.621615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:21.622499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 63/127] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:45:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:21.633420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b3831-133d-43c3-bff7-46a1df581ba2 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:21.707757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 65/128] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:21.787862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fed2d-9c52-4c23-bfa0-8e6d769d09e4 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:21.800945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b3831-133d-43c3-bff7-46a1df581ba2 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:21.876239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ebca2-db17-4520-b9f1-16a04cdea006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:21.876970 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ebca2-db17-4520-b9f1-16a04cdea006 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:21.877263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ebca2-db17-4520-b9f1-16a04cdea006 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:21.877544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ebca2-db17-4520-b9f1-16a04cdea006 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:21.877862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:21.878317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ebca2-db17-4520-b9f1-16a04cdea006 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:21.878641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ebca2-db17-4520-b9f1-16a04cdea006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:21.880032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ebca2-db17-4520-b9f1-16a04cdea006 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:21.902598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 64/129] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:45:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:21.920575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ebca2-db17-4520-b9f1-16a04cdea006 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:21.967501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fed2d-9c52-4c23-bfa0-8e6d769d09e4 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:22.018109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:22.027362 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507ac08-bf5f-47c9-b2ec-0b991a6e4f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:22.028182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507ac08-bf5f-47c9-b2ec-0b991a6e4f97 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:22.028471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507ac08-bf5f-47c9-b2ec-0b991a6e4f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:22.028774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507ac08-bf5f-47c9-b2ec-0b991a6e4f97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:22.041178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507ac08-bf5f-47c9-b2ec-0b991a6e4f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-2507ac08-bf5f-47c9-b2ec-0b991a6e4f97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:22.049948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:22.050841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:22.051627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 66/130] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:22.059442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507ac08-bf5f-47c9-b2ec-0b991a6e4f97 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:22.144698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ebca2-db17-4520-b9f1-16a04cdea006 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:22.185858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507ac08-bf5f-47c9-b2ec-0b991a6e4f97 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:22.204105 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:22.246254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b05b63-4d7d-41c8-9f83-947978a25ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:22.246937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b05b63-4d7d-41c8-9f83-947978a25ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:22.247233 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b05b63-4d7d-41c8-9f83-947978a25ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:22.247518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b05b63-4d7d-41c8-9f83-947978a25ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:22.248395 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b05b63-4d7d-41c8-9f83-947978a25ff5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:22.248711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b05b63-4d7d-41c8-9f83-947978a25ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:22.250204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b05b63-4d7d-41c8-9f83-947978a25ff5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:22.253396 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8cc9f-1bde-45e9-a078-b1ac2882b731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:22.254586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8cc9f-1bde-45e9-a078-b1ac2882b731 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:22.255095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8cc9f-1bde-45e9-a078-b1ac2882b731 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:22.255583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8cc9f-1bde-45e9-a078-b1ac2882b731 None admin] PATH_INFO: `/v3/users/944f4c75142549088d2481820e193ef8` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:22.260775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8cc9f-1bde-45e9-a078-b1ac2882b731 None admin] RBAC: Authorizing `identity:get_user(user_id=944f4c75142549088d2481820e193ef8)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:22.262780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8cc9f-1bde-45e9-a078-b1ac2882b731 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:22.264040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 67/131] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:22.264935 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 65/132] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:21 2022] GET /identity/v3/users/944f4c75142549088d2481820e193ef8 => generated 275 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:22.270427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b05b63-4d7d-41c8-9f83-947978a25ff5 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:22.273742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8cc9f-1bde-45e9-a078-b1ac2882b731 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:22.438943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b05b63-4d7d-41c8-9f83-947978a25ff5 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:22.473537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db62c93-6f2d-46d6-ac55-653d5127e868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:22.474475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db62c93-6f2d-46d6-ac55-653d5127e868 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:22.474777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db62c93-6f2d-46d6-ac55-653d5127e868 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:22.475063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db62c93-6f2d-46d6-ac55-653d5127e868 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:22.476300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db62c93-6f2d-46d6-ac55-653d5127e868 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:22.477856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db62c93-6f2d-46d6-ac55-653d5127e868 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:22.479598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 68/133] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:22 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:22.485054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db62c93-6f2d-46d6-ac55-653d5127e868 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:22.491125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8cc9f-1bde-45e9-a078-b1ac2882b731 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:22.526746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:22.539877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:22.547403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce11603-4add-47b0-bd57-8af9e0c91651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:22.548313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce11603-4add-47b0-bd57-8af9e0c91651 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:22.548611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce11603-4add-47b0-bd57-8af9e0c91651 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:22.549426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce11603-4add-47b0-bd57-8af9e0c91651 None admin] PATH_INFO: `/v3/users/944f4c75142549088d2481820e193ef8` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:22.555203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce11603-4add-47b0-bd57-8af9e0c91651 None admin] RBAC: Authorizing `identity:get_user(user_id=944f4c75142549088d2481820e193ef8)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:22.556821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce11603-4add-47b0-bd57-8af9e0c91651 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:22.558722 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 66/134] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:22 2022] GET /identity/v3/users/944f4c75142549088d2481820e193ef8 => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:22.561076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da72873-dd7e-4182-b580-2808b4e489b1 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:22.561378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da72873-dd7e-4182-b580-2808b4e489b1 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:22.561667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da72873-dd7e-4182-b580-2808b4e489b1 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:22.562974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 67/135] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:22.567675 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da72873-dd7e-4182-b580-2808b4e489b1 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:22.589404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db62c93-6f2d-46d6-ac55-653d5127e868 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:22.623712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d89a4-f31f-43f8-8e09-9b7f60fe4500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:22.624610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d89a4-f31f-43f8-8e09-9b7f60fe4500 None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:22.624900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d89a4-f31f-43f8-8e09-9b7f60fe4500 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:22.625184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d89a4-f31f-43f8-8e09-9b7f60fe4500 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/users/944f4c75142549088d2481820e193ef8/roles/c7b7d3b4dd944936851dee41ead5dc99` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:22.627980 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d89a4-f31f-43f8-8e09-9b7f60fe4500 None admin] RBAC: Authorizing `identity:create_grant(project_id=a75c6b7e46c24ae8adacf3d5669e7162, user_id=944f4c75142549088d2481820e193ef8, role_id=c7b7d3b4dd944936851dee41ead5dc99)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:22.629461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d89a4-f31f-43f8-8e09-9b7f60fe4500 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:22.648681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 69/136] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:45:22 2022] PUT /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/users/944f4c75142549088d2481820e193ef8/roles/c7b7d3b4dd944936851dee41ead5dc99 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 12:45:22.650981 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fd196-86d8-4440-8bf3-f113cd7dbdd3 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:22.651284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fd196-86d8-4440-8bf3-f113cd7dbdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:22.657512 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fd196-86d8-4440-8bf3-f113cd7dbdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:22.691538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-bf5fd196-86d8-4440-8bf3-f113cd7dbdd3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:22.693143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bf5fd196-86d8-4440-8bf3-f113cd7dbdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:22.849080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 70/137] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:22.860007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da72873-dd7e-4182-b580-2808b4e489b1 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:22.885766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fd196-86d8-4440-8bf3-f113cd7dbdd3 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:22.912618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b355a8f-2af3-41a2-a8f7-0e67ddb8e659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:22.913621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b355a8f-2af3-41a2-a8f7-0e67ddb8e659 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:22.914570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b355a8f-2af3-41a2-a8f7-0e67ddb8e659 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:22.914936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b355a8f-2af3-41a2-a8f7-0e67ddb8e659 None admin] PATH_INFO: `/v3/projects/bafc86bc275846f5861a1b426ebd7c81` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:22.916501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b355a8f-2af3-41a2-a8f7-0e67ddb8e659 None admin] RBAC: Authorizing `identity:get_project(project_id=bafc86bc275846f5861a1b426ebd7c81)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:22.918430 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b355a8f-2af3-41a2-a8f7-0e67ddb8e659 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:22.920770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 68/138] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:22 2022] GET /identity/v3/projects/bafc86bc275846f5861a1b426ebd7c81 => generated 312 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:22.927158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b355a8f-2af3-41a2-a8f7-0e67ddb8e659 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:22.941969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:22.999811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:23.050191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b355a8f-2af3-41a2-a8f7-0e67ddb8e659 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:23.103505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fd196-86d8-4440-8bf3-f113cd7dbdd3 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:23.117902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617b055-ca91-4f99-a638-79101e443899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:23.119151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617b055-ca91-4f99-a638-79101e443899 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:23.119632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617b055-ca91-4f99-a638-79101e443899 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:23.120098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617b055-ca91-4f99-a638-79101e443899 None admin] PATH_INFO: `/v3/projects/bafc86bc275846f5861a1b426ebd7c81/users/944f4c75142549088d2481820e193ef8/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:23.124066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617b055-ca91-4f99-a638-79101e443899 None admin] RBAC: Authorizing `identity:create_grant(project_id=bafc86bc275846f5861a1b426ebd7c81, user_id=944f4c75142549088d2481820e193ef8, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:23.125736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617b055-ca91-4f99-a638-79101e443899 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:23.146085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 69/139] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:45:22 2022] PUT /identity/v3/projects/bafc86bc275846f5861a1b426ebd7c81/users/944f4c75142549088d2481820e193ef8/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:23.169781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14920762-bfc1-4338-9eb5-0410f5376210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:23.171035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920762-bfc1-4338-9eb5-0410f5376210 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:23.171551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920762-bfc1-4338-9eb5-0410f5376210 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:23.172032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920762-bfc1-4338-9eb5-0410f5376210 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:23.196944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14920762-bfc1-4338-9eb5-0410f5376210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-14920762-bfc1-4338-9eb5-0410f5376210 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:23.206454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:23.207382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:23.208121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:23.210803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 71/140] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:23.216969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617b055-ca91-4f99-a638-79101e443899 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:23.364910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617b055-ca91-4f99-a638-79101e443899 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:23.390434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:23.414476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8d3d1-939a-45c8-9c6a-48b3d5c881ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:23.415172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8d3d1-939a-45c8-9c6a-48b3d5c881ba None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:23.415464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8d3d1-939a-45c8-9c6a-48b3d5c881ba None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:23.415749 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8d3d1-939a-45c8-9c6a-48b3d5c881ba None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:23.416521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8d3d1-939a-45c8-9c6a-48b3d5c881ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:23.416839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8d3d1-939a-45c8-9c6a-48b3d5c881ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:23.418224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8d3d1-939a-45c8-9c6a-48b3d5c881ba None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:23.430783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 70/141] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:23.436323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14920762-bfc1-4338-9eb5-0410f5376210 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:23.566056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:23.585094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14920762-bfc1-4338-9eb5-0410f5376210 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:23.621571 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa658b-5b11-448d-8bf0-8c181ba07288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:23.622841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa658b-5b11-448d-8bf0-8c181ba07288 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:23.623328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa658b-5b11-448d-8bf0-8c181ba07288 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:23.623802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa658b-5b11-448d-8bf0-8c181ba07288 None admin] PATH_INFO: `/v3/users/944f4c75142549088d2481820e193ef8` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:23.625216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa658b-5b11-448d-8bf0-8c181ba07288 None admin] RBAC: Authorizing `identity:get_user(user_id=944f4c75142549088d2481820e193ef8)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:23.627526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa658b-5b11-448d-8bf0-8c181ba07288 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:23.629359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 72/142] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:23 2022] GET /identity/v3/users/944f4c75142549088d2481820e193ef8 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:23.638311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8d3d1-939a-45c8-9c6a-48b3d5c881ba None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:23.786569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8d3d1-939a-45c8-9c6a-48b3d5c881ba None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:23.832353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6edb5d-5b8a-49dc-8324-cbe32d79862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:23.833020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6edb5d-5b8a-49dc-8324-cbe32d79862c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:23.833308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6edb5d-5b8a-49dc-8324-cbe32d79862c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:23.833597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6edb5d-5b8a-49dc-8324-cbe32d79862c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:23.904449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:23.934774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671346-e427-4f77-bee7-80f155458201 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:23.935419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671346-e427-4f77-bee7-80f155458201 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:23.935903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671346-e427-4f77-bee7-80f155458201 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:23.937883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 73/143] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:23.941653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6edb5d-5b8a-49dc-8324-cbe32d79862c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:23.943165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6edb5d-5b8a-49dc-8324-cbe32d79862c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:23.945662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff36ed-2077-4ff7-816c-697cae2a55a5 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:23.946184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff36ed-2077-4ff7-816c-697cae2a55a5 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:23.946694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff36ed-2077-4ff7-816c-697cae2a55a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:23.966809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-4eff36ed-2077-4ff7-816c-697cae2a55a5 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:23.969054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4eff36ed-2077-4ff7-816c-697cae2a55a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:24.050850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 74/144] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:24.080801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff36ed-2077-4ff7-816c-697cae2a55a5 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:24.151495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 71/145] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:24.156970 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6edb5d-5b8a-49dc-8324-cbe32d79862c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:24.189553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:24.208400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff36ed-2077-4ff7-816c-697cae2a55a5 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:24.253927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e8297-1b67-44d9-bbf9-0d5c63b04156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:24.255370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e8297-1b67-44d9-bbf9-0d5c63b04156 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:24.255877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e8297-1b67-44d9-bbf9-0d5c63b04156 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:24.256350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e8297-1b67-44d9-bbf9-0d5c63b04156 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:24.257768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e8297-1b67-44d9-bbf9-0d5c63b04156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:24.260444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e8297-1b67-44d9-bbf9-0d5c63b04156 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:24.262962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 75/146] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:24.269286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e8297-1b67-44d9-bbf9-0d5c63b04156 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:24.346479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:24.380809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e8297-1b67-44d9-bbf9-0d5c63b04156 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:24.384265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6edb5d-5b8a-49dc-8324-cbe32d79862c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:24.429771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a49bd-bd3f-4752-b0f4-8e8280e5edd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:24.430944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a49bd-bd3f-4752-b0f4-8e8280e5edd5 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:24.431480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a49bd-bd3f-4752-b0f4-8e8280e5edd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:24.431950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a49bd-bd3f-4752-b0f4-8e8280e5edd5 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:24.432883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a49bd-bd3f-4752-b0f4-8e8280e5edd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:24.434583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a49bd-bd3f-4752-b0f4-8e8280e5edd5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:24.459148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68823416-2131-46f7-b112-f20a2fd383d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:24.460460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68823416-2131-46f7-b112-f20a2fd383d4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:24.461023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68823416-2131-46f7-b112-f20a2fd383d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:24.461497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68823416-2131-46f7-b112-f20a2fd383d4 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:24.463241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68823416-2131-46f7-b112-f20a2fd383d4 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:24.465213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68823416-2131-46f7-b112-f20a2fd383d4 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:24.474906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 76/147] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:45:24 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:24.475249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 72/148] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:24 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:24.479319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a49bd-bd3f-4752-b0f4-8e8280e5edd5 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:24.491135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fa8ad-9f13-46fa-b0d7-366dbe02589f None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:24.491747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fa8ad-9f13-46fa-b0d7-366dbe02589f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:24.492257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fa8ad-9f13-46fa-b0d7-366dbe02589f None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:24.494254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 73/149] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:24.502947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:24.510313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e085-075b-40cb-ac0f-cf555ddda5ef None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:24.510313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e085-075b-40cb-ac0f-cf555ddda5ef None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:24.510313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e085-075b-40cb-ac0f-cf555ddda5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:24.546915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e2b6e085-075b-40cb-ac0f-cf555ddda5ef None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:24.548151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e2b6e085-075b-40cb-ac0f-cf555ddda5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:24.690428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 74/150] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:24.696138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a49bd-bd3f-4752-b0f4-8e8280e5edd5 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:24.714182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6e085-075b-40cb-ac0f-cf555ddda5ef None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:24.769659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f3bbb-8834-4fea-9284-7d8eb1994d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:24.772173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f3bbb-8834-4fea-9284-7d8eb1994d52 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:24.772475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f3bbb-8834-4fea-9284-7d8eb1994d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:24.772757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f3bbb-8834-4fea-9284-7d8eb1994d52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:24.773608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f3bbb-8834-4fea-9284-7d8eb1994d52 None admin] RBAC: Adding query filter params (role.id=27d8d0ea0b83488a95a9141d439247f6, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162, user.id=944f4c75142549088d2481820e193ef8) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:24.773923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f3bbb-8834-4fea-9284-7d8eb1994d52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:24.775410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f3bbb-8834-4fea-9284-7d8eb1994d52 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:24.789537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 77/151] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:24 2022] GET /identity/v3/role_assignments?user.id=944f4c75142549088d2481820e193ef8&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:24.829420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:24.863317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6e085-075b-40cb-ac0f-cf555ddda5ef None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:24.909689 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec8e51-e526-40e2-bfd4-cd79268988c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:24.910603 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec8e51-e526-40e2-bfd4-cd79268988c1 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:24.910886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec8e51-e526-40e2-bfd4-cd79268988c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:24.911161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec8e51-e526-40e2-bfd4-cd79268988c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:24.926664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec8e51-e526-40e2-bfd4-cd79268988c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-d2ec8e51-e526-40e2-bfd4-cd79268988c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:24.934479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:24.935409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:24.936179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 75/152] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:45:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:24.941727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f3bbb-8834-4fea-9284-7d8eb1994d52 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:24.962976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6c819-0f33-4ae5-a410-e0e9d7b639c8 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:24.963310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6c819-0f33-4ae5-a410-e0e9d7b639c8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:24.963529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6c819-0f33-4ae5-a410-e0e9d7b639c8 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:24.965154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 76/153] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:24.972473 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776788f-d057-4b87-805a-e1fe986d8769 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:24.972846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776788f-d057-4b87-805a-e1fe986d8769 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:24.973123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776788f-d057-4b87-805a-e1fe986d8769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:25.007790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-8776788f-d057-4b87-805a-e1fe986d8769 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:25.008814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8776788f-d057-4b87-805a-e1fe986d8769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:25.056782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:25.112896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f3bbb-8834-4fea-9284-7d8eb1994d52 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:25.143629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 77/154] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:25.164958 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f65fc-33e5-4765-b833-1f81d26bcebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:25.165977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f65fc-33e5-4765-b833-1f81d26bcebb None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:25.166581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f65fc-33e5-4765-b833-1f81d26bcebb None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:25.167059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f65fc-33e5-4765-b833-1f81d26bcebb None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:25.168503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776788f-d057-4b87-805a-e1fe986d8769 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:25.171137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f65fc-33e5-4765-b833-1f81d26bcebb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:25.171750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f65fc-33e5-4765-b833-1f81d26bcebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:25.173620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f65fc-33e5-4765-b833-1f81d26bcebb None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:25.184032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 78/155] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:25.190000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f65fc-33e5-4765-b833-1f81d26bcebb None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:25.257230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:25.303033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776788f-d057-4b87-805a-e1fe986d8769 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:25.334147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f65fc-33e5-4765-b833-1f81d26bcebb None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:25.372288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7827c-ee1f-4cca-ae5a-1eb3034671e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:25.373009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7827c-ee1f-4cca-ae5a-1eb3034671e5 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:25.373370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7827c-ee1f-4cca-ae5a-1eb3034671e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:25.373744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7827c-ee1f-4cca-ae5a-1eb3034671e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:25.378310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0091e44-8937-41d6-992c-3c1cbfd81a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:25.379543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0091e44-8937-41d6-992c-3c1cbfd81a83 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:25.380136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0091e44-8937-41d6-992c-3c1cbfd81a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:25.380593 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0091e44-8937-41d6-992c-3c1cbfd81a83 None admin] PATH_INFO: `/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:25.382494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0091e44-8937-41d6-992c-3c1cbfd81a83 None admin] RBAC: Authorizing `identity:get_user(user_id=c0168c20cbe64bb7a4d17219c4fe3eef)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:25.384214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0091e44-8937-41d6-992c-3c1cbfd81a83 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:25.389121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 79/156] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:25 2022] GET /identity/v3/users/c0168c20cbe64bb7a4d17219c4fe3eef => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:25.395082 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7827c-ee1f-4cca-ae5a-1eb3034671e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:25.395538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0091e44-8937-41d6-992c-3c1cbfd81a83 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:25.407548 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-e4a7827c-ee1f-4cca-ae5a-1eb3034671e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:25.410314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:25.411267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:25.412064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:25.414569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 78/157] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:25.425038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7827c-ee1f-4cca-ae5a-1eb3034671e5 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:25.515917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0091e44-8937-41d6-992c-3c1cbfd81a83 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:25.561096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e6c05-ce2d-4caa-9b6c-0d7e83cbbb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:25.561769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e6c05-ce2d-4caa-9b6c-0d7e83cbbb86 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:25.562055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e6c05-ce2d-4caa-9b6c-0d7e83cbbb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:25.562352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e6c05-ce2d-4caa-9b6c-0d7e83cbbb86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:25.637300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7827c-ee1f-4cca-ae5a-1eb3034671e5 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:25.690931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:25.693471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e6c05-ce2d-4caa-9b6c-0d7e83cbbb86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:25.697086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e6c05-ce2d-4caa-9b6c-0d7e83cbbb86 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:25.711667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357940b-f4ed-4d20-9a3a-d1f9b8500d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:25.712357 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357940b-f4ed-4d20-9a3a-d1f9b8500d92 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:25.712642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357940b-f4ed-4d20-9a3a-d1f9b8500d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:25.713107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357940b-f4ed-4d20-9a3a-d1f9b8500d92 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:25.713916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357940b-f4ed-4d20-9a3a-d1f9b8500d92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:25.714225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357940b-f4ed-4d20-9a3a-d1f9b8500d92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:25.714672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:25.715657 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357940b-f4ed-4d20-9a3a-d1f9b8500d92 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:25.727037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 79/158] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:25.732984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357940b-f4ed-4d20-9a3a-d1f9b8500d92 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:25.853721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 80/159] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:25.859327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e6c05-ce2d-4caa-9b6c-0d7e83cbbb86 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:25.922610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357940b-f4ed-4d20-9a3a-d1f9b8500d92 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:25.985707 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7555904-4524-4055-9e5e-9eca040fa284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:25.986951 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7555904-4524-4055-9e5e-9eca040fa284 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:25.987395 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7555904-4524-4055-9e5e-9eca040fa284 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:25.989388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7555904-4524-4055-9e5e-9eca040fa284 None admin] PATH_INFO: `/v3/users/1e91129d27284f81931cb58522ffc41e` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:25.990842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7555904-4524-4055-9e5e-9eca040fa284 None admin] RBAC: Authorizing `identity:get_user(user_id=1e91129d27284f81931cb58522ffc41e)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:25.992452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7555904-4524-4055-9e5e-9eca040fa284 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:25.994384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 80/160] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:25 2022] GET /identity/v3/users/1e91129d27284f81931cb58522ffc41e => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:26.000058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7555904-4524-4055-9e5e-9eca040fa284 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:26.045979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e6c05-ce2d-4caa-9b6c-0d7e83cbbb86 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:26.051315 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:26.066913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:26.132441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aad2be-cb0a-47e8-9a39-6b22f9c99ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:26.133226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aad2be-cb0a-47e8-9a39-6b22f9c99ddd None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:26.133533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aad2be-cb0a-47e8-9a39-6b22f9c99ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:26.133782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aad2be-cb0a-47e8-9a39-6b22f9c99ddd None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:26.136596 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7555904-4524-4055-9e5e-9eca040fa284 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:26.139055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aad2be-cb0a-47e8-9a39-6b22f9c99ddd None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:26.140539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aad2be-cb0a-47e8-9a39-6b22f9c99ddd None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:26.143168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 81/161] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:25 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:26.159624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aad2be-cb0a-47e8-9a39-6b22f9c99ddd None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:26.210793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd04d60-9b2f-4fe2-82c2-7377730872f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:26.213445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd04d60-9b2f-4fe2-82c2-7377730872f4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:26.214126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd04d60-9b2f-4fe2-82c2-7377730872f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:26.214865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd04d60-9b2f-4fe2-82c2-7377730872f4 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:26.218158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd04d60-9b2f-4fe2-82c2-7377730872f4 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:26.221265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd04d60-9b2f-4fe2-82c2-7377730872f4 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:26.224631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 81/162] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:25 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:26.230349 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd04d60-9b2f-4fe2-82c2-7377730872f4 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:26.390658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd04d60-9b2f-4fe2-82c2-7377730872f4 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:26.412478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aad2be-cb0a-47e8-9a39-6b22f9c99ddd None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:26.440677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:26.461707 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25045ef-92f6-40b5-9d3d-72cdd0a3927e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:26.462901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25045ef-92f6-40b5-9d3d-72cdd0a3927e None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:26.463324 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25045ef-92f6-40b5-9d3d-72cdd0a3927e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:26.463763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25045ef-92f6-40b5-9d3d-72cdd0a3927e None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/users/1e91129d27284f81931cb58522ffc41e/roles/4a71e72752894223a2b3fd53f728c211` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:26.468152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25045ef-92f6-40b5-9d3d-72cdd0a3927e None admin] RBAC: Authorizing `identity:create_grant(project_id=a75c6b7e46c24ae8adacf3d5669e7162, user_id=1e91129d27284f81931cb58522ffc41e, role_id=4a71e72752894223a2b3fd53f728c211)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:26.469778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25045ef-92f6-40b5-9d3d-72cdd0a3927e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:26.490087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57743cff-c3bf-42f8-bae5-3d7166d59a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:26.490863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57743cff-c3bf-42f8-bae5-3d7166d59a16 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:26.491161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57743cff-c3bf-42f8-bae5-3d7166d59a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:26.491424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57743cff-c3bf-42f8-bae5-3d7166d59a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:26.492278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57743cff-c3bf-42f8-bae5-3d7166d59a16 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162, user.id=c0168c20cbe64bb7a4d17219c4fe3eef) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:26.492592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57743cff-c3bf-42f8-bae5-3d7166d59a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:26.494053 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57743cff-c3bf-42f8-bae5-3d7166d59a16 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:26.497733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 82/163] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:45:26 2022] PUT /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/users/1e91129d27284f81931cb58522ffc41e/roles/4a71e72752894223a2b3fd53f728c211 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:26.505821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 82/164] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:26 2022] GET /identity/v3/role_assignments?user.id=c0168c20cbe64bb7a4d17219c4fe3eef&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=0c473678b281495f86f58b077be04e1b => generated 608 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:26.528525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca2b6c-9fbe-4403-a577-1616c8360586 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:26.529048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca2b6c-9fbe-4403-a577-1616c8360586 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:26.529556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca2b6c-9fbe-4403-a577-1616c8360586 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:26.531176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 83/165] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:26.537678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76587db-fc08-42da-8489-84de0f2b1024 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:26.537972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76587db-fc08-42da-8489-84de0f2b1024 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:26.538259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76587db-fc08-42da-8489-84de0f2b1024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:26.559363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a76587db-fc08-42da-8489-84de0f2b1024 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:26.560647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a76587db-fc08-42da-8489-84de0f2b1024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:26.602963 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:26.659991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 83/166] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:26.695432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca2b6c-9fbe-4403-a577-1616c8360586 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:26.812889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a0a1-6b96-46ef-ba33-b3c8ae7e4025 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:26.813237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a0a1-6b96-46ef-ba33-b3c8ae7e4025 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:26.813470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a0a1-6b96-46ef-ba33-b3c8ae7e4025 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:26.815426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 84/167] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:45:26.821347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b56dfe-61ef-45af-aa0d-4950ca61e5d9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:26.821643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b56dfe-61ef-45af-aa0d-4950ca61e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:26.821922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b56dfe-61ef-45af-aa0d-4950ca61e5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:26.851187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca2b6c-9fbe-4403-a577-1616c8360586 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:26.871438 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-49b56dfe-61ef-45af-aa0d-4950ca61e5d9 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:26.872674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-49b56dfe-61ef-45af-aa0d-4950ca61e5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:26.910469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40713b6f-070f-4460-b355-1291b7b9766f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:26.911808 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40713b6f-070f-4460-b355-1291b7b9766f None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:26.912292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40713b6f-070f-4460-b355-1291b7b9766f None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:26.912765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40713b6f-070f-4460-b355-1291b7b9766f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:26.926219 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40713b6f-070f-4460-b355-1291b7b9766f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-40713b6f-070f-4460-b355-1291b7b9766f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:26.935066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:26.936106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:26.936894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:26.939724 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 84/168] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:26.945523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40713b6f-070f-4460-b355-1291b7b9766f None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:27.018385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:27.038022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 85/169] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:27.065931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b56dfe-61ef-45af-aa0d-4950ca61e5d9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:27.126903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40713b6f-070f-4460-b355-1291b7b9766f None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:27.174783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:27.186575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b56dfe-61ef-45af-aa0d-4950ca61e5d9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:27.191605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300dbf6-3701-460c-90cd-e8b10863c1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:27.192303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300dbf6-3701-460c-90cd-e8b10863c1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:27.192588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300dbf6-3701-460c-90cd-e8b10863c1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:27.192863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300dbf6-3701-460c-90cd-e8b10863c1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:27.193627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300dbf6-3701-460c-90cd-e8b10863c1d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:27.193937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300dbf6-3701-460c-90cd-e8b10863c1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:27.195352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300dbf6-3701-460c-90cd-e8b10863c1d8 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:27.208243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 85/170] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:27.216107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300dbf6-3701-460c-90cd-e8b10863c1d8 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:27.242752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d83e8-1f15-4ceb-9484-f1dd06daf9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:27.243952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d83e8-1f15-4ceb-9484-f1dd06daf9ef None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:27.244429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d83e8-1f15-4ceb-9484-f1dd06daf9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:27.244945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d83e8-1f15-4ceb-9484-f1dd06daf9ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:27.260844 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d83e8-1f15-4ceb-9484-f1dd06daf9ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-b29d83e8-1f15-4ceb-9484-f1dd06daf9ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:27.276831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:27.277732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:27.278625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:27.281275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 86/171] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:45:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:27.287536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d83e8-1f15-4ceb-9484-f1dd06daf9ef None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:27.496177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d83e8-1f15-4ceb-9484-f1dd06daf9ef None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:27.496780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300dbf6-3701-460c-90cd-e8b10863c1d8 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:27.508437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:27.559407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd128c60-3630-4233-852b-038d570e933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:27.560211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd128c60-3630-4233-852b-038d570e933d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:27.560587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd128c60-3630-4233-852b-038d570e933d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:27.560870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd128c60-3630-4233-852b-038d570e933d None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:27.561634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd128c60-3630-4233-852b-038d570e933d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:27.561943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd128c60-3630-4233-852b-038d570e933d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:27.563535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd128c60-3630-4233-852b-038d570e933d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:27.566448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7032a2a-6ff3-479b-adc1-ff119bda1fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:27.567351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7032a2a-6ff3-479b-adc1-ff119bda1fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:27.567639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7032a2a-6ff3-479b-adc1-ff119bda1fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:27.567922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7032a2a-6ff3-479b-adc1-ff119bda1fc9 None admin] PATH_INFO: `/v3/users/944f4c75142549088d2481820e193ef8` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:27.570161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7032a2a-6ff3-479b-adc1-ff119bda1fc9 None admin] RBAC: Authorizing `identity:get_user(user_id=944f4c75142549088d2481820e193ef8)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:27.571628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7032a2a-6ff3-479b-adc1-ff119bda1fc9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:27.573773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 86/172] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:27 2022] GET /identity/v3/users/944f4c75142549088d2481820e193ef8 => generated 275 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:27.576155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 87/173] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:45:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:27.579031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7032a2a-6ff3-479b-adc1-ff119bda1fc9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:27.582953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd128c60-3630-4233-852b-038d570e933d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:27.627711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:27.769816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7032a2a-6ff3-479b-adc1-ff119bda1fc9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:27.797033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd128c60-3630-4233-852b-038d570e933d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:27.807394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae9a4b-d9cf-44d9-9339-24e8f5cde5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:27.808064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae9a4b-d9cf-44d9-9339-24e8f5cde5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:27.808350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae9a4b-d9cf-44d9-9339-24e8f5cde5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:27.808630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae9a4b-d9cf-44d9-9339-24e8f5cde5d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:27.855578 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:27.882145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae9a4b-d9cf-44d9-9339-24e8f5cde5d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:27.883522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a7ee2-2859-4b4c-9dc2-8793e1333f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:27.883652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae9a4b-d9cf-44d9-9339-24e8f5cde5d0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:27.884410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a7ee2-2859-4b4c-9dc2-8793e1333f4c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:27.884692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a7ee2-2859-4b4c-9dc2-8793e1333f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:27.884968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a7ee2-2859-4b4c-9dc2-8793e1333f4c None admin] PATH_INFO: `/v3/users/944f4c75142549088d2481820e193ef8` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:27.891058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a7ee2-2859-4b4c-9dc2-8793e1333f4c None admin] RBAC: Authorizing `identity:get_user(user_id=944f4c75142549088d2481820e193ef8)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:27.893098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a7ee2-2859-4b4c-9dc2-8793e1333f4c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:27.895147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 88/174] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:27 2022] GET /identity/v3/users/944f4c75142549088d2481820e193ef8 => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:27.903162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a7ee2-2859-4b4c-9dc2-8793e1333f4c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:28.046067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 87/175] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:45:28.054575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae9a4b-d9cf-44d9-9339-24e8f5cde5d0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:28.110687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:28.133348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a7ee2-2859-4b4c-9dc2-8793e1333f4c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:28.185865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:28.192454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a147dc0-22dd-4a1c-b1e8-6af5c5f0a1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:28.193123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a147dc0-22dd-4a1c-b1e8-6af5c5f0a1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:28.193410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a147dc0-22dd-4a1c-b1e8-6af5c5f0a1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:28.193687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a147dc0-22dd-4a1c-b1e8-6af5c5f0a1b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:28.212102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae9a4b-d9cf-44d9-9339-24e8f5cde5d0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:28.248079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265d98a-c9a8-4f45-bde2-2ef4c1930f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:28.248949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265d98a-c9a8-4f45-bde2-2ef4c1930f89 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:28.249233 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265d98a-c9a8-4f45-bde2-2ef4c1930f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:28.249504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265d98a-c9a8-4f45-bde2-2ef4c1930f89 None admin] PATH_INFO: `/v3/projects/bafc86bc275846f5861a1b426ebd7c81` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:28.250757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265d98a-c9a8-4f45-bde2-2ef4c1930f89 None admin] RBAC: Authorizing `identity:get_project(project_id=bafc86bc275846f5861a1b426ebd7c81)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:28.252737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265d98a-c9a8-4f45-bde2-2ef4c1930f89 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:28.254464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 88/176] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:28 2022] GET /identity/v3/projects/bafc86bc275846f5861a1b426ebd7c81 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:28.267601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265d98a-c9a8-4f45-bde2-2ef4c1930f89 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:28.331621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a147dc0-22dd-4a1c-b1e8-6af5c5f0a1b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:28.333173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a147dc0-22dd-4a1c-b1e8-6af5c5f0a1b7 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:28.469723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265d98a-c9a8-4f45-bde2-2ef4c1930f89 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:28.518728 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:28.545877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 89/177] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:45:28.546408 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58f93e-c1e1-4623-98e1-bd1caa79f429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:28.547082 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58f93e-c1e1-4623-98e1-bd1caa79f429 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:28.547365 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58f93e-c1e1-4623-98e1-bd1caa79f429 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:28.547642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58f93e-c1e1-4623-98e1-bd1caa79f429 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:28.549001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58f93e-c1e1-4623-98e1-bd1caa79f429 None admin] RBAC: Adding query filter params (role.id=27d8d0ea0b83488a95a9141d439247f6, scope.project.id=bafc86bc275846f5861a1b426ebd7c81, user.id=944f4c75142549088d2481820e193ef8) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:28.549317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58f93e-c1e1-4623-98e1-bd1caa79f429 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:28.550793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58f93e-c1e1-4623-98e1-bd1caa79f429 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:28.557944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a147dc0-22dd-4a1c-b1e8-6af5c5f0a1b7 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:28.563352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 89/178] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:28 2022] GET /identity/v3/role_assignments?user.id=944f4c75142549088d2481820e193ef8&scope.project.id=bafc86bc275846f5861a1b426ebd7c81&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:28.716460 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a147dc0-22dd-4a1c-b1e8-6af5c5f0a1b7 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:28.736435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac2add-fb33-4f49-8d27-a3959e78b876 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:28.736777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac2add-fb33-4f49-8d27-a3959e78b876 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:28.737002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac2add-fb33-4f49-8d27-a3959e78b876 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:28.742470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 90/179] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:45:28.747831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f388-4985-41f9-a50a-c0393a0067c0 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:28.748123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f388-4985-41f9-a50a-c0393a0067c0 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:28.748402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f388-4985-41f9-a50a-c0393a0067c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:28.768670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a2f16-c295-432e-8dbc-2222fcbcb74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:28.769186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:28.769919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a2f16-c295-432e-8dbc-2222fcbcb74b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:28.770202 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a2f16-c295-432e-8dbc-2222fcbcb74b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:28.770503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a2f16-c295-432e-8dbc-2222fcbcb74b None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:28.778351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a2f16-c295-432e-8dbc-2222fcbcb74b None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:28.778351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a2f16-c295-432e-8dbc-2222fcbcb74b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:28.778755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 90/180] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:28 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:28.789981 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1209f388-4985-41f9-a50a-c0393a0067c0 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:28.790158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a2f16-c295-432e-8dbc-2222fcbcb74b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:28.791736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1209f388-4985-41f9-a50a-c0393a0067c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:28.890681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:28.916239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 91/181] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:28.942648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209f388-4985-41f9-a50a-c0393a0067c0 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:28.989674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a2f16-c295-432e-8dbc-2222fcbcb74b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:29.053115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61871820-5873-4517-bdf9-646e5662d234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:29.053782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871820-5873-4517-bdf9-646e5662d234 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.054064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871820-5873-4517-bdf9-646e5662d234 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.054358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871820-5873-4517-bdf9-646e5662d234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.055282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61871820-5873-4517-bdf9-646e5662d234 None admin] RBAC: Adding query filter params (role.id=c7b7d3b4dd944936851dee41ead5dc99, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162, user.id=944f4c75142549088d2481820e193ef8) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:29.055594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61871820-5873-4517-bdf9-646e5662d234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:29.059588 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61871820-5873-4517-bdf9-646e5662d234 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:29.072021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 91/182] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:28 2022] GET /identity/v3/role_assignments?user.id=944f4c75142549088d2481820e193ef8&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=c7b7d3b4dd944936851dee41ead5dc99 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:29.138072 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209f388-4985-41f9-a50a-c0393a0067c0 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:29.178645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd28a3-bc39-498f-a430-7f7abc5079d0 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.178974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd28a3-bc39-498f-a430-7f7abc5079d0 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.179206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd28a3-bc39-498f-a430-7f7abc5079d0 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.180859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 92/183] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:45:29.185995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51da0b-d0fc-4dae-b0d1-3ddb654b155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:29.186898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51da0b-d0fc-4dae-b0d1-3ddb654b155d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.187185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51da0b-d0fc-4dae-b0d1-3ddb654b155d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.187466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51da0b-d0fc-4dae-b0d1-3ddb654b155d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.192099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f99821-035c-46d6-b7df-38f162a030be None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.192400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f99821-035c-46d6-b7df-38f162a030be None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.192651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f99821-035c-46d6-b7df-38f162a030be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.195502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:29.218701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-36f99821-035c-46d6-b7df-38f162a030be None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:29.219916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-36f99821-035c-46d6-b7df-38f162a030be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:29.221726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51da0b-d0fc-4dae-b0d1-3ddb654b155d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-ca51da0b-d0fc-4dae-b0d1-3ddb654b155d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:29.229694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:29.230669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:29.231463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:29.232804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 92/184] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:45:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:29.242065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51da0b-d0fc-4dae-b0d1-3ddb654b155d None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:29.359361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 93/185] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:29.389534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f99821-035c-46d6-b7df-38f162a030be None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:29.418499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51da0b-d0fc-4dae-b0d1-3ddb654b155d None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:29.458126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:29.467492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b31683-98f8-4f72-a102-584e568684f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:29.468207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b31683-98f8-4f72-a102-584e568684f8 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.468494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b31683-98f8-4f72-a102-584e568684f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.468767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b31683-98f8-4f72-a102-584e568684f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.469534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b31683-98f8-4f72-a102-584e568684f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:29.469841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b31683-98f8-4f72-a102-584e568684f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:29.471878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b31683-98f8-4f72-a102-584e568684f8 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:29.481023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 93/186] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:29.488901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c917f7-f955-40d7-86a9-6c7fe922d121 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.488901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c917f7-f955-40d7-86a9-6c7fe922d121 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.488901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c917f7-f955-40d7-86a9-6c7fe922d121 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.488901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 94/187] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:29.494252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d696c2-1796-4c39-87f8-a1899ef81024 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.494501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d696c2-1796-4c39-87f8-a1899ef81024 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.494778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d696c2-1796-4c39-87f8-a1899ef81024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.514338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-14d696c2-1796-4c39-87f8-a1899ef81024 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:29.515241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-14d696c2-1796-4c39-87f8-a1899ef81024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:29.532575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f99821-035c-46d6-b7df-38f162a030be None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:29.568357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4531539-1e97-4944-912d-6c9e6db213f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:29.569624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4531539-1e97-4944-912d-6c9e6db213f3 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.569911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4531539-1e97-4944-912d-6c9e6db213f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.570489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4531539-1e97-4944-912d-6c9e6db213f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.575646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:29.583615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4531539-1e97-4944-912d-6c9e6db213f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:29.598188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 95/188] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:29.598188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d696c2-1796-4c39-87f8-a1899ef81024 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-f4531539-1e97-4944-912d-6c9e6db213f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:29.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:29.599459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:29.600417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 94/189] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:45:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:29.602474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4531539-1e97-4944-912d-6c9e6db213f3 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:29.699800 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d696c2-1796-4c39-87f8-a1899ef81024 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:29.704267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4531539-1e97-4944-912d-6c9e6db213f3 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:29.713374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:29.733185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9775de2-d87e-4688-a6a4-84d3fd027bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:29.734063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9775de2-d87e-4688-a6a4-84d3fd027bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.734369 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9775de2-d87e-4688-a6a4-84d3fd027bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.734640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9775de2-d87e-4688-a6a4-84d3fd027bf4 None admin] PATH_INFO: `/v3/users/cd13ff46a6e14dc68ee9f695b590cf66` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.737942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cba99c-699b-48d9-8bd4-6fdb3128eeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:29.738635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cba99c-699b-48d9-8bd4-6fdb3128eeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.738923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cba99c-699b-48d9-8bd4-6fdb3128eeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.739202 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cba99c-699b-48d9-8bd4-6fdb3128eeb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.739956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cba99c-699b-48d9-8bd4-6fdb3128eeb3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:29.740267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cba99c-699b-48d9-8bd4-6fdb3128eeb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:29.741770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cba99c-699b-48d9-8bd4-6fdb3128eeb3 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:29.747254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9775de2-d87e-4688-a6a4-84d3fd027bf4 None admin] RBAC: Authorizing `identity:get_user(user_id=cd13ff46a6e14dc68ee9f695b590cf66)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:29.748644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9775de2-d87e-4688-a6a4-84d3fd027bf4 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:29.749506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 95/190] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:45:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:29.755160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 96/191] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:29 2022] GET /identity/v3/users/cd13ff46a6e14dc68ee9f695b590cf66 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:29.755160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec04eb-51dc-47ee-8507-4a24e26a4f34 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.755160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec04eb-51dc-47ee-8507-4a24e26a4f34 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.755160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec04eb-51dc-47ee-8507-4a24e26a4f34 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.755160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 97/192] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:29.755591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cba99c-699b-48d9-8bd4-6fdb3128eeb3 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:29.758410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec04eb-51dc-47ee-8507-4a24e26a4f34 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:29.804231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:29.851273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cba99c-699b-48d9-8bd4-6fdb3128eeb3 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:29.857123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec04eb-51dc-47ee-8507-4a24e26a4f34 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:29.884318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8867e-5038-4775-8672-e7913b1ae081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:29.885100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8867e-5038-4775-8672-e7913b1ae081 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.885384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8867e-5038-4775-8672-e7913b1ae081 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.885659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8867e-5038-4775-8672-e7913b1ae081 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.886834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8867e-5038-4775-8672-e7913b1ae081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:29.888575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8867e-5038-4775-8672-e7913b1ae081 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:29.890351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 96/193] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:29.890884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8867e-5038-4775-8672-e7913b1ae081 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:29.890973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9931d2-b315-4b92-953c-cc68f38ec2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:29.891860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9931d2-b315-4b92-953c-cc68f38ec2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.892174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9931d2-b315-4b92-953c-cc68f38ec2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.892477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9931d2-b315-4b92-953c-cc68f38ec2b0 None admin] PATH_INFO: `/v3/users/cd13ff46a6e14dc68ee9f695b590cf66` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.893664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9931d2-b315-4b92-953c-cc68f38ec2b0 None admin] RBAC: Authorizing `identity:get_user(user_id=cd13ff46a6e14dc68ee9f695b590cf66)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:29.895454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9931d2-b315-4b92-953c-cc68f38ec2b0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:29.897179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 98/194] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:29 2022] GET /identity/v3/users/cd13ff46a6e14dc68ee9f695b590cf66 => generated 283 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:29.898348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14186d7-581c-4d5b-a223-961741bb0bdd None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:29.898640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14186d7-581c-4d5b-a223-961741bb0bdd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:29.898921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14186d7-581c-4d5b-a223-961741bb0bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:29.914700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a14186d7-581c-4d5b-a223-961741bb0bdd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:29.915883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a14186d7-581c-4d5b-a223-961741bb0bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:29.929305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:29.980128 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 99/195] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:29.980992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14186d7-581c-4d5b-a223-961741bb0bdd None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:29.990624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8867e-5038-4775-8672-e7913b1ae081 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:30.024983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8899c3b-a4fa-44ae-b9c2-29bc0393e42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:30.025150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:30.025563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899c3b-a4fa-44ae-b9c2-29bc0393e42a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:30.025868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899c3b-a4fa-44ae-b9c2-29bc0393e42a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:30.026123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899c3b-a4fa-44ae-b9c2-29bc0393e42a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:30.077053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14186d7-581c-4d5b-a223-961741bb0bdd None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:30.092263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8899c3b-a4fa-44ae-b9c2-29bc0393e42a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:30.093742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8899c3b-a4fa-44ae-b9c2-29bc0393e42a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:30.109326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc2890-eb20-4098-9e7c-4461cae8c22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:30.109993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc2890-eb20-4098-9e7c-4461cae8c22e None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:30.110274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc2890-eb20-4098-9e7c-4461cae8c22e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:30.110591 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc2890-eb20-4098-9e7c-4461cae8c22e None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:30.111812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc2890-eb20-4098-9e7c-4461cae8c22e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:30.113223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc2890-eb20-4098-9e7c-4461cae8c22e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:30.131391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:30.133461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 100/196] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:45:29 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:30.134812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc2890-eb20-4098-9e7c-4461cae8c22e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:30.204542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 97/197] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:30.205746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8899c3b-a4fa-44ae-b9c2-29bc0393e42a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:30.250087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:30.253356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc2890-eb20-4098-9e7c-4461cae8c22e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:30.299516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8316f5-c559-40d7-8ff5-f3960992628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:30.300216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8316f5-c559-40d7-8ff5-f3960992628d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:30.300501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8316f5-c559-40d7-8ff5-f3960992628d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:30.300899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8316f5-c559-40d7-8ff5-f3960992628d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:30.319777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8899c3b-a4fa-44ae-b9c2-29bc0393e42a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:30.367942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbe6eb-a689-4eef-b1ff-aa780d62d627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:30.368754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbe6eb-a689-4eef-b1ff-aa780d62d627 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:30.369034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbe6eb-a689-4eef-b1ff-aa780d62d627 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:30.369303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbe6eb-a689-4eef-b1ff-aa780d62d627 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:30.377504 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:30.381750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbe6eb-a689-4eef-b1ff-aa780d62d627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-5edbe6eb-a689-4eef-b1ff-aa780d62d627 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:30.390270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:30.391400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:30.392959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:30.393710 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 98/198] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:30.394536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbe6eb-a689-4eef-b1ff-aa780d62d627 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:30.397419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8316f5-c559-40d7-8ff5-f3960992628d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:30.399034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8316f5-c559-40d7-8ff5-f3960992628d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:30.513230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbe6eb-a689-4eef-b1ff-aa780d62d627 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:30.521015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:30.524347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 101/199] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:30.525642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8316f5-c559-40d7-8ff5-f3960992628d None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:30.566241 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b010fd3-83bf-4757-a887-871d5d838be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:30.567186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b010fd3-83bf-4757-a887-871d5d838be3 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:30.567471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b010fd3-83bf-4757-a887-871d5d838be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:30.567744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b010fd3-83bf-4757-a887-871d5d838be3 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:30.568960 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b010fd3-83bf-4757-a887-871d5d838be3 None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:30.571038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b010fd3-83bf-4757-a887-871d5d838be3 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:30.573863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 99/200] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:30 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:30.574489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b010fd3-83bf-4757-a887-871d5d838be3 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:30.641420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8316f5-c559-40d7-8ff5-f3960992628d None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:30.665216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:30.681059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5392a0a-4e1a-47f6-8641-fef932d3c725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:30.682219 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5392a0a-4e1a-47f6-8641-fef932d3c725 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:30.682706 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5392a0a-4e1a-47f6-8641-fef932d3c725 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:30.683306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5392a0a-4e1a-47f6-8641-fef932d3c725 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:30.684295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5392a0a-4e1a-47f6-8641-fef932d3c725 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:30.684782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5392a0a-4e1a-47f6-8641-fef932d3c725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:30.686599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5392a0a-4e1a-47f6-8641-fef932d3c725 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:30.696321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 102/201] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:30.697668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5392a0a-4e1a-47f6-8641-fef932d3c725 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:30.708164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b010fd3-83bf-4757-a887-871d5d838be3 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:30.750195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8339291-4579-4c69-93f8-31ce7b1cbf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:30.751530 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8339291-4579-4c69-93f8-31ce7b1cbf92 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:30.752068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8339291-4579-4c69-93f8-31ce7b1cbf92 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:30.752569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8339291-4579-4c69-93f8-31ce7b1cbf92 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:30.753984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8339291-4579-4c69-93f8-31ce7b1cbf92 None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:30.755781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8339291-4579-4c69-93f8-31ce7b1cbf92 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:30.757768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 100/202] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:30 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:30.758813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8339291-4579-4c69-93f8-31ce7b1cbf92 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:30.779014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:30.831678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5392a0a-4e1a-47f6-8641-fef932d3c725 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:30.873494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8242a60-3d5e-4843-9f65-661b04f171af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:30.874324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8339291-4579-4c69-93f8-31ce7b1cbf92 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:30.874768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8242a60-3d5e-4843-9f65-661b04f171af None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:30.875275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8242a60-3d5e-4843-9f65-661b04f171af None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:30.875860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8242a60-3d5e-4843-9f65-661b04f171af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:30.877101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8242a60-3d5e-4843-9f65-661b04f171af None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=197b0ba2357a405ca5849973bb209b46, user.id=cd13ff46a6e14dc68ee9f695b590cf66) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:30.877544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8242a60-3d5e-4843-9f65-661b04f171af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:30.879296 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8242a60-3d5e-4843-9f65-661b04f171af None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:30.885861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 103/203] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:30 2022] GET /identity/v3/role_assignments?user.id=cd13ff46a6e14dc68ee9f695b590cf66&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=0c473678b281495f86f58b077be04e1b => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:30.886591 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8242a60-3d5e-4843-9f65-661b04f171af None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:30.918778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2af501-f8c2-478b-b056-8008a2dc5f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:30.920928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2af501-f8c2-478b-b056-8008a2dc5f51 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:30.921487 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2af501-f8c2-478b-b056-8008a2dc5f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:30.921957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2af501-f8c2-478b-b056-8008a2dc5f51 None admin] PATH_INFO: `/v3/users/1e91129d27284f81931cb58522ffc41e` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:30.925088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2af501-f8c2-478b-b056-8008a2dc5f51 None admin] RBAC: Authorizing `identity:get_user(user_id=1e91129d27284f81931cb58522ffc41e)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:30.927096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2af501-f8c2-478b-b056-8008a2dc5f51 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:30.930198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 101/204] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:30 2022] GET /identity/v3/users/1e91129d27284f81931cb58522ffc41e => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:30.937848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2af501-f8c2-478b-b056-8008a2dc5f51 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:30.942760 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:31.007883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8242a60-3d5e-4843-9f65-661b04f171af None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:31.037832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:31.048548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e91db-3686-40f3-8e0b-2877f8321b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:31.049242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e91db-3686-40f3-8e0b-2877f8321b31 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:31.049541 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e91db-3686-40f3-8e0b-2877f8321b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:31.049816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e91db-3686-40f3-8e0b-2877f8321b31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:31.050743 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e91db-3686-40f3-8e0b-2877f8321b31 None admin] RBAC: Adding query filter params (role.id=c7b7d3b4dd944936851dee41ead5dc99, scope.project.id=197b0ba2357a405ca5849973bb209b46, user.id=cd13ff46a6e14dc68ee9f695b590cf66) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:31.051046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e91db-3686-40f3-8e0b-2877f8321b31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:31.052786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e91db-3686-40f3-8e0b-2877f8321b31 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:31.059589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 104/205] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:30 2022] GET /identity/v3/role_assignments?user.id=cd13ff46a6e14dc68ee9f695b590cf66&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=c7b7d3b4dd944936851dee41ead5dc99 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:31.085867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2af501-f8c2-478b-b056-8008a2dc5f51 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:31.141268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a756498-7d63-4d50-a657-90b64821a433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:31.141949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a756498-7d63-4d50-a657-90b64821a433 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:31.142318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a756498-7d63-4d50-a657-90b64821a433 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:31.142609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a756498-7d63-4d50-a657-90b64821a433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:31.232772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a756498-7d63-4d50-a657-90b64821a433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:31.234313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a756498-7d63-4d50-a657-90b64821a433 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:31.236504 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:31.343976 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 102/206] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:31.349834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e91db-3686-40f3-8e0b-2877f8321b31 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:31.484461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e91db-3686-40f3-8e0b-2877f8321b31 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:31.519951 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f095e4-23dd-4ed6-84d3-815864a52580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:31.520827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f095e4-23dd-4ed6-84d3-815864a52580 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:31.521116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f095e4-23dd-4ed6-84d3-815864a52580 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:31.521634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f095e4-23dd-4ed6-84d3-815864a52580 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:31.522624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f095e4-23dd-4ed6-84d3-815864a52580 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:31.524070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f095e4-23dd-4ed6-84d3-815864a52580 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:31.525934 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 105/207] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:31 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:31.531566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a756498-7d63-4d50-a657-90b64821a433 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:31.665221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a756498-7d63-4d50-a657-90b64821a433 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:31.667637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:31.706120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38204231-528b-4c0b-a20f-ed48b1f6e63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:31.707170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204231-528b-4c0b-a20f-ed48b1f6e63d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:31.707654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204231-528b-4c0b-a20f-ed48b1f6e63d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:31.708149 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204231-528b-4c0b-a20f-ed48b1f6e63d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:31.709188 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38204231-528b-4c0b-a20f-ed48b1f6e63d None admin] RBAC: Adding query filter params (role.id=4a71e72752894223a2b3fd53f728c211, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162, user.id=1e91129d27284f81931cb58522ffc41e) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:31.709698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38204231-528b-4c0b-a20f-ed48b1f6e63d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:31.713909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38204231-528b-4c0b-a20f-ed48b1f6e63d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:31.723338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 103/208] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:31 2022] GET /identity/v3/role_assignments?user.id=1e91129d27284f81931cb58522ffc41e&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=4a71e72752894223a2b3fd53f728c211 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:32.797255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195858-9cde-40f7-99a3-4dc69179db44 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:32.797746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195858-9cde-40f7-99a3-4dc69179db44 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:32.797746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195858-9cde-40f7-99a3-4dc69179db44 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:32.799375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 106/209] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:32.804800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a432ebd-f737-4f54-9486-b9ac10fa4159 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:32.805576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a432ebd-f737-4f54-9486-b9ac10fa4159 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:32.806044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a432ebd-f737-4f54-9486-b9ac10fa4159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:32.823480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6a432ebd-f737-4f54-9486-b9ac10fa4159 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:32.824924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6a432ebd-f737-4f54-9486-b9ac10fa4159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:32.833263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644d951-37fc-4d3a-a3a2-aca3419dafd6 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:32.833550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644d951-37fc-4d3a-a3a2-aca3419dafd6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:32.834088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644d951-37fc-4d3a-a3a2-aca3419dafd6 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:32.835709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 107/210] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:45:32.842122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381c995-16e9-4857-bbc4-4d0ad2b49693 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:32.842445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381c995-16e9-4857-bbc4-4d0ad2b49693 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:32.842927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381c995-16e9-4857-bbc4-4d0ad2b49693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:32.860071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-6381c995-16e9-4857-bbc4-4d0ad2b49693 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:32.862050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6381c995-16e9-4857-bbc4-4d0ad2b49693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:32.912577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 104/211] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:32.927658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:32.936321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a432ebd-f737-4f54-9486-b9ac10fa4159 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:32.950708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 108/212] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:32.975677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381c995-16e9-4857-bbc4-4d0ad2b49693 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:33.065748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:33.072854 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a432ebd-f737-4f54-9486-b9ac10fa4159 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:33.106457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381c995-16e9-4857-bbc4-4d0ad2b49693 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:33.113752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9a148-f15d-4fab-8150-383f17a6ab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:33.114852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9a148-f15d-4fab-8150-383f17a6ab33 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:33.115341 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9a148-f15d-4fab-8150-383f17a6ab33 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:33.115846 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9a148-f15d-4fab-8150-383f17a6ab33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:33.126751 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9a148-f15d-4fab-8150-383f17a6ab33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-6ce9a148-f15d-4fab-8150-383f17a6ab33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:33.134744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:33.135617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:33.136412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 105/213] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:33.145695 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9a148-f15d-4fab-8150-383f17a6ab33 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:33.150244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf9554-c365-4b67-9dd5-49d608c58ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:33.151352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf9554-c365-4b67-9dd5-49d608c58ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:33.151832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf9554-c365-4b67-9dd5-49d608c58ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:33.152298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf9554-c365-4b67-9dd5-49d608c58ad6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:33.153690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf9554-c365-4b67-9dd5-49d608c58ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:33.155372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf9554-c365-4b67-9dd5-49d608c58ad6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:33.157532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 109/214] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:33.164816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf9554-c365-4b67-9dd5-49d608c58ad6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:33.226734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:33.256595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9a148-f15d-4fab-8150-383f17a6ab33 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:33.281679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf9554-c365-4b67-9dd5-49d608c58ad6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:33.303548 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df2082-a82d-4879-9ec9-be3a94ccde86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:33.304612 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df2082-a82d-4879-9ec9-be3a94ccde86 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:33.304969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df2082-a82d-4879-9ec9-be3a94ccde86 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:33.305314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df2082-a82d-4879-9ec9-be3a94ccde86 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:33.306432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df2082-a82d-4879-9ec9-be3a94ccde86 None admin] RBAC: Adding query filter params (name=member) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:33.306818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df2082-a82d-4879-9ec9-be3a94ccde86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:33.308701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df2082-a82d-4879-9ec9-be3a94ccde86 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:33.325114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 106/215] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:33.326631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df2082-a82d-4879-9ec9-be3a94ccde86 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:33.330699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a99980-ecc1-4f42-a403-05e2364c176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:33.332452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a99980-ecc1-4f42-a403-05e2364c176e None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:33.332886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a99980-ecc1-4f42-a403-05e2364c176e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:33.333304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a99980-ecc1-4f42-a403-05e2364c176e None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:33.334171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a99980-ecc1-4f42-a403-05e2364c176e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:33.335815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a99980-ecc1-4f42-a403-05e2364c176e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:33.360831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:33.373728 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 110/216] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:45:33 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:33.452921 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df2082-a82d-4879-9ec9-be3a94ccde86 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:33.472813 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b240b-b8c4-4c71-8840-949cea3f2e8b None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:33.473126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b240b-b8c4-4c71-8840-949cea3f2e8b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:33.473392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b240b-b8c4-4c71-8840-949cea3f2e8b None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:33.475238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 111/217] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:45:33.481160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bad52-ca4a-4d88-a9b3-db5c85128385 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:33.481469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bad52-ca4a-4d88-a9b3-db5c85128385 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:33.481735 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bad52-ca4a-4d88-a9b3-db5c85128385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:33.497188 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65c57c-3527-46c7-9a69-2026842e970f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:33.498204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65c57c-3527-46c7-9a69-2026842e970f None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:33.498830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65c57c-3527-46c7-9a69-2026842e970f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:33.499115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65c57c-3527-46c7-9a69-2026842e970f None admin] PATH_INFO: `/v3/users/f1acc21bb0b344db8bf6a6d5ce64b724` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:33.504426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-652bad52-ca4a-4d88-a9b3-db5c85128385 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:33.505698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-652bad52-ca4a-4d88-a9b3-db5c85128385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:33.517843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65c57c-3527-46c7-9a69-2026842e970f None admin] RBAC: Authorizing `identity:get_user(user_id=f1acc21bb0b344db8bf6a6d5ce64b724)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:33.519869 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65c57c-3527-46c7-9a69-2026842e970f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:33.522712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 107/218] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:33 2022] GET /identity/v3/users/f1acc21bb0b344db8bf6a6d5ce64b724 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:33.527999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65c57c-3527-46c7-9a69-2026842e970f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:33.593052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 112/219] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:33.611781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:33.615556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bad52-ca4a-4d88-a9b3-db5c85128385 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:33.637909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65c57c-3527-46c7-9a69-2026842e970f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:33.673208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:33.686657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8747c-bebc-4c67-9dc3-0790bd391bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:33.686657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8747c-bebc-4c67-9dc3-0790bd391bac None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:33.686657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8747c-bebc-4c67-9dc3-0790bd391bac None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:33.686657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8747c-bebc-4c67-9dc3-0790bd391bac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:33.755433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bad52-ca4a-4d88-a9b3-db5c85128385 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:33.771293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8747c-bebc-4c67-9dc3-0790bd391bac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:33.772776 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8747c-bebc-4c67-9dc3-0790bd391bac None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:33.808227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8d098-2ccc-4b37-89d0-06ac05736004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:33.809160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8d098-2ccc-4b37-89d0-06ac05736004 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:33.810015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8d098-2ccc-4b37-89d0-06ac05736004 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:33.810015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8d098-2ccc-4b37-89d0-06ac05736004 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:33.821222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b8d098-2ccc-4b37-89d0-06ac05736004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-48b8d098-2ccc-4b37-89d0-06ac05736004 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:33.829398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:33.830980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:33.831794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 113/220] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:45:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:33.833201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084cc78-e7cd-4d24-9b73-a08e3b4380d9 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:33.833201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084cc78-e7cd-4d24-9b73-a08e3b4380d9 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:33.833201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084cc78-e7cd-4d24-9b73-a08e3b4380d9 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:33.835029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 114/221] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:33.839157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084cc78-e7cd-4d24-9b73-a08e3b4380d9 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:33.866904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:33.892889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 108/222] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:45:33.894223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af84dec-081c-48be-81e8-1aa61721053f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:33.894545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af84dec-081c-48be-81e8-1aa61721053f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:33.894837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af84dec-081c-48be-81e8-1aa61721053f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:33.911116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6af84dec-081c-48be-81e8-1aa61721053f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:33.912485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6af84dec-081c-48be-81e8-1aa61721053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:33.949863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084cc78-e7cd-4d24-9b73-a08e3b4380d9 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:33.951548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:33.983363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 109/223] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:33.984041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af84dec-081c-48be-81e8-1aa61721053f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:33.999229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a520248-101e-40a9-97a4-c58e3624edac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:34.000200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a520248-101e-40a9-97a4-c58e3624edac None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:34.000516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a520248-101e-40a9-97a4-c58e3624edac None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:34.000799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a520248-101e-40a9-97a4-c58e3624edac None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:34.002100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a520248-101e-40a9-97a4-c58e3624edac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:34.002526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a520248-101e-40a9-97a4-c58e3624edac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:34.004012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a520248-101e-40a9-97a4-c58e3624edac None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:34.018417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 115/224] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:34.024777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a520248-101e-40a9-97a4-c58e3624edac None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:34.092311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af84dec-081c-48be-81e8-1aa61721053f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:34.100568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:34.129612 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0559e-ef1b-4a58-b84e-fa91ac991ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:34.130408 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0559e-ef1b-4a58-b84e-fa91ac991ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:34.130693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0559e-ef1b-4a58-b84e-fa91ac991ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:34.130972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0559e-ef1b-4a58-b84e-fa91ac991ce5 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:34.132411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0559e-ef1b-4a58-b84e-fa91ac991ce5 None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:34.133822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0559e-ef1b-4a58-b84e-fa91ac991ce5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:34.135945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 110/225] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:33 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:34.136591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0559e-ef1b-4a58-b84e-fa91ac991ce5 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:34.139001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a520248-101e-40a9-97a4-c58e3624edac None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:34.178331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770895de-d163-4926-910c-ab577ad0b997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:34.179477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770895de-d163-4926-910c-ab577ad0b997 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:34.179899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770895de-d163-4926-910c-ab577ad0b997 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:34.180325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770895de-d163-4926-910c-ab577ad0b997 None admin] PATH_INFO: `/v3/users/cd13ff46a6e14dc68ee9f695b590cf66` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:34.181658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770895de-d163-4926-910c-ab577ad0b997 None admin] RBAC: Authorizing `identity:get_user(user_id=cd13ff46a6e14dc68ee9f695b590cf66)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:34.183324 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770895de-d163-4926-910c-ab577ad0b997 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:34.185104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 116/226] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:34 2022] GET /identity/v3/users/cd13ff46a6e14dc68ee9f695b590cf66 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:34.186065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770895de-d163-4926-910c-ab577ad0b997 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:34.230984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:34.252019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0559e-ef1b-4a58-b84e-fa91ac991ce5 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:34.289376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4c97b-e282-481b-a730-cbf68d518824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:34.290256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4c97b-e282-481b-a730-cbf68d518824 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:34.290707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4c97b-e282-481b-a730-cbf68d518824 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:34.290992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4c97b-e282-481b-a730-cbf68d518824 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:34.302122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4c97b-e282-481b-a730-cbf68d518824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-1aa4c97b-e282-481b-a730-cbf68d518824 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:34.310441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:34.311360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:34.312135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 111/227] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:45:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:34.312801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4c97b-e282-481b-a730-cbf68d518824 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:34.323767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770895de-d163-4926-910c-ab577ad0b997 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:34.354107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:34.372161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f227a-c056-467f-b44a-6fc29c1000cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:34.373061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f227a-c056-467f-b44a-6fc29c1000cc None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:34.373345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f227a-c056-467f-b44a-6fc29c1000cc None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:34.373622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f227a-c056-467f-b44a-6fc29c1000cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:34.374526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f227a-c056-467f-b44a-6fc29c1000cc None admin] RBAC: Adding query filter params (role.id=27d8d0ea0b83488a95a9141d439247f6, scope.project.id=197b0ba2357a405ca5849973bb209b46, user.id=f1acc21bb0b344db8bf6a6d5ce64b724) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:34.374841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f227a-c056-467f-b44a-6fc29c1000cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:34.376288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f227a-c056-467f-b44a-6fc29c1000cc None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:34.382984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 117/228] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:34 2022] GET /identity/v3/role_assignments?user.id=f1acc21bb0b344db8bf6a6d5ce64b724&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:34.383748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f227a-c056-467f-b44a-6fc29c1000cc None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:34.450654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4c97b-e282-481b-a730-cbf68d518824 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:34.489534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:34.499442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f227a-c056-467f-b44a-6fc29c1000cc None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:34.501518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6557bfd-7aaa-4114-9f6e-1be2f7511559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:34.503732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6557bfd-7aaa-4114-9f6e-1be2f7511559 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:34.504293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6557bfd-7aaa-4114-9f6e-1be2f7511559 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:34.504805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6557bfd-7aaa-4114-9f6e-1be2f7511559 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:34.506571 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6557bfd-7aaa-4114-9f6e-1be2f7511559 None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:34.509196 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6557bfd-7aaa-4114-9f6e-1be2f7511559 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:34.512222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 112/229] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:34 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:34.518876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6557bfd-7aaa-4114-9f6e-1be2f7511559 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:34.540182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff9955-c95c-4aa3-b309-bc7ad6267ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:34.540927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff9955-c95c-4aa3-b309-bc7ad6267ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:34.541181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff9955-c95c-4aa3-b309-bc7ad6267ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:34.541461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff9955-c95c-4aa3-b309-bc7ad6267ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:34.542259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff9955-c95c-4aa3-b309-bc7ad6267ff6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:34.542648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff9955-c95c-4aa3-b309-bc7ad6267ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:34.544045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff9955-c95c-4aa3-b309-bc7ad6267ff6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:34.554038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 118/230] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:45:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:34.559478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff9955-c95c-4aa3-b309-bc7ad6267ff6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:34.632091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6557bfd-7aaa-4114-9f6e-1be2f7511559 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:34.634191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:34.671645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea9f08-4f80-424b-803e-5dfc8451f9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:34.672922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea9f08-4f80-424b-803e-5dfc8451f9db None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:34.673768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff9955-c95c-4aa3-b309-bc7ad6267ff6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:34.674166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea9f08-4f80-424b-803e-5dfc8451f9db None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:34.674698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea9f08-4f80-424b-803e-5dfc8451f9db None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46/users/cd13ff46a6e14dc68ee9f695b590cf66/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:34.676987 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea9f08-4f80-424b-803e-5dfc8451f9db None admin] RBAC: Authorizing `identity:create_grant(project_id=197b0ba2357a405ca5849973bb209b46, user_id=cd13ff46a6e14dc68ee9f695b590cf66, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:34.678743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea9f08-4f80-424b-803e-5dfc8451f9db None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:34.696157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 113/231] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:45:34 2022] PUT /identity/v3/projects/197b0ba2357a405ca5849973bb209b46/users/cd13ff46a6e14dc68ee9f695b590cf66/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:34.716194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0590e0-ed1e-4773-8943-5465a3d3670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:34.717217 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0590e0-ed1e-4773-8943-5465a3d3670d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:34.717500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0590e0-ed1e-4773-8943-5465a3d3670d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:34.717872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0590e0-ed1e-4773-8943-5465a3d3670d None admin] PATH_INFO: `/v3/users/cd13ff46a6e14dc68ee9f695b590cf66` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:34.719210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0590e0-ed1e-4773-8943-5465a3d3670d None admin] RBAC: Authorizing `identity:get_user(user_id=cd13ff46a6e14dc68ee9f695b590cf66)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:34.721336 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0590e0-ed1e-4773-8943-5465a3d3670d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:34.723333 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 119/232] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:34 2022] GET /identity/v3/users/cd13ff46a6e14dc68ee9f695b590cf66 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:34.729105 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea9f08-4f80-424b-803e-5dfc8451f9db None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:34.848123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea9f08-4f80-424b-803e-5dfc8451f9db None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:34.885953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcfff9-fef6-4e1c-9fca-54873030b47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:34.887064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcfff9-fef6-4e1c-9fca-54873030b47c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:34.887358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcfff9-fef6-4e1c-9fca-54873030b47c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:34.887641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcfff9-fef6-4e1c-9fca-54873030b47c None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:34.889037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcfff9-fef6-4e1c-9fca-54873030b47c None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:34.890623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcfff9-fef6-4e1c-9fca-54873030b47c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:34.892569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 114/233] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:34 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:34.899712 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0590e0-ed1e-4773-8943-5465a3d3670d None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:34.946307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:35.038029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0590e0-ed1e-4773-8943-5465a3d3670d None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:35.087029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e315e06-df05-468d-8c0a-8cbb4e1039d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:35.088177 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e315e06-df05-468d-8c0a-8cbb4e1039d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:35.088637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e315e06-df05-468d-8c0a-8cbb4e1039d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:35.089063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e315e06-df05-468d-8c0a-8cbb4e1039d1 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46/users/cd13ff46a6e14dc68ee9f695b590cf66/roles/c7b7d3b4dd944936851dee41ead5dc99` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:35.091486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e315e06-df05-468d-8c0a-8cbb4e1039d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=197b0ba2357a405ca5849973bb209b46, user_id=cd13ff46a6e14dc68ee9f695b590cf66, role_id=c7b7d3b4dd944936851dee41ead5dc99)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:35.093092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e315e06-df05-468d-8c0a-8cbb4e1039d1 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:35.118480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 120/234] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:45:34 2022] PUT /identity/v3/projects/197b0ba2357a405ca5849973bb209b46/users/cd13ff46a6e14dc68ee9f695b590cf66/roles/c7b7d3b4dd944936851dee41ead5dc99 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:36.115230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1444c-ca9b-4720-bf26-f9fc2488040c None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:36.115737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1444c-ca9b-4720-bf26-f9fc2488040c None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:36.115812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1444c-ca9b-4720-bf26-f9fc2488040c None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:36.117405 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 115/235] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:36.122798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bacbef-8e51-45a7-aafb-211289111ff4 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:36.123584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bacbef-8e51-45a7-aafb-211289111ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:36.124063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bacbef-8e51-45a7-aafb-211289111ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:36.145113 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-13bacbef-8e51-45a7-aafb-211289111ff4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:36.146593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-13bacbef-8e51-45a7-aafb-211289111ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:36.235510 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 121/236] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:36.260746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1444c-ca9b-4720-bf26-f9fc2488040c None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:36.292327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:36.304451 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78202-6e8a-4945-8c07-a9dbb5ed59d6 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:36.304742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78202-6e8a-4945-8c07-a9dbb5ed59d6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:36.305025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78202-6e8a-4945-8c07-a9dbb5ed59d6 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:36.306594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 122/237] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:36.311526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edc5c4-7eb6-448c-b954-52f9b7822dbd None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:36.311838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edc5c4-7eb6-448c-b954-52f9b7822dbd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:36.312180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edc5c4-7eb6-448c-b954-52f9b7822dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:36.330809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e5edc5c4-7eb6-448c-b954-52f9b7822dbd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:36.333435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e5edc5c4-7eb6-448c-b954-52f9b7822dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:36.348972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:36.390355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1444c-ca9b-4720-bf26-f9fc2488040c None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:36.414746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 123/238] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:36.431556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1114b-d290-4ed3-be52-486f5e60228e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:36.432596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1114b-d290-4ed3-be52-486f5e60228e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:36.433024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1114b-d290-4ed3-be52-486f5e60228e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:36.433441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1114b-d290-4ed3-be52-486f5e60228e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:36.434769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1114b-d290-4ed3-be52-486f5e60228e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:36.436375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1114b-d290-4ed3-be52-486f5e60228e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:36.438439 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 116/239] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:36.446330 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edc5c4-7eb6-448c-b954-52f9b7822dbd None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:36.449735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1114b-d290-4ed3-be52-486f5e60228e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:36.547506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:36.562690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1114b-d290-4ed3-be52-486f5e60228e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:36.594565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edc5c4-7eb6-448c-b954-52f9b7822dbd None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:36.613283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d1cc2-eae3-4438-a5b1-ce9c498eeb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:36.614580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d1cc2-eae3-4438-a5b1-ce9c498eeb0e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:36.615194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d1cc2-eae3-4438-a5b1-ce9c498eeb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:36.615670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d1cc2-eae3-4438-a5b1-ce9c498eeb0e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:36.626830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d1cc2-eae3-4438-a5b1-ce9c498eeb0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-f91d1cc2-eae3-4438-a5b1-ce9c498eeb0e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:36.635034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:36.635901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:36.636716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:36.639595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 117/240] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:36.646044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d1cc2-eae3-4438-a5b1-ce9c498eeb0e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:36.650989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d2637-283f-4add-a89c-a18b94618085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:36.651678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d2637-283f-4add-a89c-a18b94618085 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:36.651968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d2637-283f-4add-a89c-a18b94618085 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:36.652249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d2637-283f-4add-a89c-a18b94618085 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:36.652987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d2637-283f-4add-a89c-a18b94618085 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:36.654484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d2637-283f-4add-a89c-a18b94618085 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:36.692889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 124/241] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:45:36 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:36.759429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d1cc2-eae3-4438-a5b1-ce9c498eeb0e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:36.796716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381edf6-f420-4f8e-aec3-22e7afd51a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:36.797781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381edf6-f420-4f8e-aec3-22e7afd51a6e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:36.798876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381edf6-f420-4f8e-aec3-22e7afd51a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:36.799354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381edf6-f420-4f8e-aec3-22e7afd51a6e None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:36.800336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381edf6-f420-4f8e-aec3-22e7afd51a6e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:36.800870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381edf6-f420-4f8e-aec3-22e7afd51a6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:36.802579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381edf6-f420-4f8e-aec3-22e7afd51a6e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:36.812887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 118/242] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:36.818984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d2637-283f-4add-a89c-a18b94618085 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:36.822554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:36.962550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d2637-283f-4add-a89c-a18b94618085 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:37.022531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13d89f-0908-447a-898c-11cb595962d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:37.023429 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13d89f-0908-447a-898c-11cb595962d0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:37.023711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13d89f-0908-447a-898c-11cb595962d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:37.023986 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13d89f-0908-447a-898c-11cb595962d0 None admin] PATH_INFO: `/v3/users/67a76e9e72e045769a678e7b96f8dbf3` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:37.048570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13d89f-0908-447a-898c-11cb595962d0 None admin] RBAC: Authorizing `identity:get_user(user_id=67a76e9e72e045769a678e7b96f8dbf3)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:37.050044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13d89f-0908-447a-898c-11cb595962d0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:37.052158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 125/243] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:36 2022] GET /identity/v3/users/67a76e9e72e045769a678e7b96f8dbf3 => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:37.057366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381edf6-f420-4f8e-aec3-22e7afd51a6e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:37.087265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:37.175494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381edf6-f420-4f8e-aec3-22e7afd51a6e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:37.213839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eed9a9-50d5-494b-b5fa-d01e1e5da5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:37.214460 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eed9a9-50d5-494b-b5fa-d01e1e5da5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:37.214752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eed9a9-50d5-494b-b5fa-d01e1e5da5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:37.215032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eed9a9-50d5-494b-b5fa-d01e1e5da5a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:37.290316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eed9a9-50d5-494b-b5fa-d01e1e5da5a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:37.291889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eed9a9-50d5-494b-b5fa-d01e1e5da5a0 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:37.314371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:37.346736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d7dcb-d0f4-4a4d-b388-f55c935d8dba None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:37.348087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d7dcb-d0f4-4a4d-b388-f55c935d8dba None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:37.348868 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d7dcb-d0f4-4a4d-b388-f55c935d8dba None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:37.350523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 126/244] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:37.357067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751fff9-1674-4e83-9157-ba7a3b172d64 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:37.357660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751fff9-1674-4e83-9157-ba7a3b172d64 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:37.358133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751fff9-1674-4e83-9157-ba7a3b172d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:37.383911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5751fff9-1674-4e83-9157-ba7a3b172d64 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:37.385418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5751fff9-1674-4e83-9157-ba7a3b172d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:37.419546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 119/245] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:37.427935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eed9a9-50d5-494b-b5fa-d01e1e5da5a0 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:37.474361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:37.487963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 127/246] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:37.489303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318ab3-e78d-4a28-92d0-940b6c4de30d None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:37.489599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318ab3-e78d-4a28-92d0-940b6c4de30d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:37.489883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318ab3-e78d-4a28-92d0-940b6c4de30d None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:37.495464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 128/247] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:37.511602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec770-9217-459c-b199-20d3b5acf908 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:37.512222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec770-9217-459c-b199-20d3b5acf908 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:37.512428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec770-9217-459c-b199-20d3b5acf908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:37.531485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-aedec770-9217-459c-b199-20d3b5acf908 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:37.532704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-aedec770-9217-459c-b199-20d3b5acf908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:37.561860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eed9a9-50d5-494b-b5fa-d01e1e5da5a0 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:37.588109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:37.607828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdafc34-6dc5-49e0-9dcc-f78f2ff8bbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:37.608697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdafc34-6dc5-49e0-9dcc-f78f2ff8bbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:37.608980 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdafc34-6dc5-49e0-9dcc-f78f2ff8bbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:37.609253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdafc34-6dc5-49e0-9dcc-f78f2ff8bbd6 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:37.611719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdafc34-6dc5-49e0-9dcc-f78f2ff8bbd6 None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:37.613362 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdafc34-6dc5-49e0-9dcc-f78f2ff8bbd6 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:37.616182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 120/248] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:37 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:37.617374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdafc34-6dc5-49e0-9dcc-f78f2ff8bbd6 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:37.621833 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 129/249] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:37.627448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedec770-9217-459c-b199-20d3b5acf908 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:37.786917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdafc34-6dc5-49e0-9dcc-f78f2ff8bbd6 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:37.795261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedec770-9217-459c-b199-20d3b5acf908 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:37.835393 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:37.848878 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831dd3-9a70-407d-8864-d163f3596baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:37.849704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831dd3-9a70-407d-8864-d163f3596baa None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:37.849985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831dd3-9a70-407d-8864-d163f3596baa None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:37.850259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831dd3-9a70-407d-8864-d163f3596baa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:37.865452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a6f57-2e4d-49dd-bdce-101b4b62216f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:37.866124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a6f57-2e4d-49dd-bdce-101b4b62216f None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:37.866525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a6f57-2e4d-49dd-bdce-101b4b62216f None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:37.866807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a6f57-2e4d-49dd-bdce-101b4b62216f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:37.867649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a6f57-2e4d-49dd-bdce-101b4b62216f None admin] RBAC: Adding query filter params (role.id=4a71e72752894223a2b3fd53f728c211, scope.project.id=197b0ba2357a405ca5849973bb209b46, user.id=67a76e9e72e045769a678e7b96f8dbf3) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:37.867964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a6f57-2e4d-49dd-bdce-101b4b62216f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:37.869422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a6f57-2e4d-49dd-bdce-101b4b62216f None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:37.869657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc831dd3-9a70-407d-8864-d163f3596baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:37.875171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 130/250] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:37 2022] GET /identity/v3/role_assignments?user.id=67a76e9e72e045769a678e7b96f8dbf3&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=4a71e72752894223a2b3fd53f728c211 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:37.875971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a6f57-2e4d-49dd-bdce-101b4b62216f None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-dc831dd3-9a70-407d-8864-d163f3596baa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:37.880924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:37.881786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:37.883361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 121/251] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:37.896364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831dd3-9a70-407d-8864-d163f3596baa None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:37.994693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:38.107646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831dd3-9a70-407d-8864-d163f3596baa None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:38.129065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a6f57-2e4d-49dd-bdce-101b4b62216f None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:38.182953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5b48-c3cf-4bb6-aa36-94fc1270f129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:38.183629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5b48-c3cf-4bb6-aa36-94fc1270f129 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:38.183911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5b48-c3cf-4bb6-aa36-94fc1270f129 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:38.184186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5b48-c3cf-4bb6-aa36-94fc1270f129 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:38.184932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5b48-c3cf-4bb6-aa36-94fc1270f129 None admin] RBAC: Adding query filter params (name=member) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:38.185238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5b48-c3cf-4bb6-aa36-94fc1270f129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:38.186670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5b48-c3cf-4bb6-aa36-94fc1270f129 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:38.191344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee00913-9fd5-416c-8320-454a0bd472d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:38.192421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee00913-9fd5-416c-8320-454a0bd472d4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:38.192921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee00913-9fd5-416c-8320-454a0bd472d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:38.193402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee00913-9fd5-416c-8320-454a0bd472d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:38.200417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 122/252] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:38.201782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6c842-7808-4955-9b8b-98f040fbb63e None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:38.202091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6c842-7808-4955-9b8b-98f040fbb63e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:38.202540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6c842-7808-4955-9b8b-98f040fbb63e None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:38.203859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 123/253] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:38.206389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6c842-7808-4955-9b8b-98f040fbb63e None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:38.206665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee00913-9fd5-416c-8320-454a0bd472d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-7ee00913-9fd5-416c-8320-454a0bd472d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:38.224777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:38.225726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:38.226567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:38.229278 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 131/254] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:45:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:38.230835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d772484-56ae-402e-b9fa-d7f5728247dd None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:38.231331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d772484-56ae-402e-b9fa-d7f5728247dd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:38.231809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d772484-56ae-402e-b9fa-d7f5728247dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:38.267715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-0d772484-56ae-402e-b9fa-d7f5728247dd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:38.270778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-0d772484-56ae-402e-b9fa-d7f5728247dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:38.337836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:38.396477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6c842-7808-4955-9b8b-98f040fbb63e None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:38.399034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 132/255] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:38.405235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d772484-56ae-402e-b9fa-d7f5728247dd None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:38.434922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:38.525501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250d25b-083c-4324-8248-0d92b465af5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:38.526401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250d25b-083c-4324-8248-0d92b465af5c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:38.526685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250d25b-083c-4324-8248-0d92b465af5c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:38.526959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250d25b-083c-4324-8248-0d92b465af5c None admin] PATH_INFO: `/v3/users/f1acc21bb0b344db8bf6a6d5ce64b724` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:38.528228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250d25b-083c-4324-8248-0d92b465af5c None admin] RBAC: Authorizing `identity:get_user(user_id=f1acc21bb0b344db8bf6a6d5ce64b724)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:38.529676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250d25b-083c-4324-8248-0d92b465af5c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:38.531320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 124/256] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:38 2022] GET /identity/v3/users/f1acc21bb0b344db8bf6a6d5ce64b724 => generated 297 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:38.541111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250d25b-083c-4324-8248-0d92b465af5c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:38.551783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d772484-56ae-402e-b9fa-d7f5728247dd None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:38.626102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de1a28-6a9c-42ec-9018-8e18ad43d0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:38.627710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de1a28-6a9c-42ec-9018-8e18ad43d0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:38.628200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de1a28-6a9c-42ec-9018-8e18ad43d0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:38.628685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de1a28-6a9c-42ec-9018-8e18ad43d0a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:38.629677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de1a28-6a9c-42ec-9018-8e18ad43d0a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:38.630186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de1a28-6a9c-42ec-9018-8e18ad43d0a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:38.631801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de1a28-6a9c-42ec-9018-8e18ad43d0a1 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:38.653786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 133/257] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:38.653786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de1a28-6a9c-42ec-9018-8e18ad43d0a1 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:38.662754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250d25b-083c-4324-8248-0d92b465af5c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:38.703197 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e4691-0914-44aa-a377-6c10caf29dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:38.704073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e4691-0914-44aa-a377-6c10caf29dee None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:38.705209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e4691-0914-44aa-a377-6c10caf29dee None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:38.705595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e4691-0914-44aa-a377-6c10caf29dee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:38.730310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e4691-0914-44aa-a377-6c10caf29dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-c27e4691-0914-44aa-a377-6c10caf29dee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:38.738830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:38.739732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:38.740519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:38.744336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 125/258] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:45:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:38.745205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e4691-0914-44aa-a377-6c10caf29dee None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:38.750498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:38.796305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:38.869333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e4691-0914-44aa-a377-6c10caf29dee None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:38.921852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de1a28-6a9c-42ec-9018-8e18ad43d0a1 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:38.933961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba5d4e-e9bb-4338-a099-86e9dea7159c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:38.935015 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba5d4e-e9bb-4338-a099-86e9dea7159c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:38.935282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba5d4e-e9bb-4338-a099-86e9dea7159c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:38.935553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba5d4e-e9bb-4338-a099-86e9dea7159c None admin] PATH_INFO: `/v3/users/cd13ff46a6e14dc68ee9f695b590cf66` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:38.938579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba5d4e-e9bb-4338-a099-86e9dea7159c None admin] RBAC: Authorizing `identity:get_user(user_id=cd13ff46a6e14dc68ee9f695b590cf66)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:38.940226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba5d4e-e9bb-4338-a099-86e9dea7159c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:38.942208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 126/259] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:38 2022] GET /identity/v3/users/cd13ff46a6e14dc68ee9f695b590cf66 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:38.943137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba5d4e-e9bb-4338-a099-86e9dea7159c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:38.992441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091586c-5bf7-4636-9931-6c7a3fd9509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:38.993664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091586c-5bf7-4636-9931-6c7a3fd9509b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:38.994153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091586c-5bf7-4636-9931-6c7a3fd9509b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:38.994777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091586c-5bf7-4636-9931-6c7a3fd9509b None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:38.996295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091586c-5bf7-4636-9931-6c7a3fd9509b None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:38.997962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091586c-5bf7-4636-9931-6c7a3fd9509b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:38.999949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 134/260] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:38 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:39.000909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091586c-5bf7-4636-9931-6c7a3fd9509b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:39.063699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba5d4e-e9bb-4338-a099-86e9dea7159c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:39.071697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:39.101229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970284d-b3d3-4186-9f94-4334aa2cb03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:39.101905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970284d-b3d3-4186-9f94-4334aa2cb03b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:39.102189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970284d-b3d3-4186-9f94-4334aa2cb03b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:39.102500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970284d-b3d3-4186-9f94-4334aa2cb03b None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:39.103234 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970284d-b3d3-4186-9f94-4334aa2cb03b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:39.103614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970284d-b3d3-4186-9f94-4334aa2cb03b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:39.105254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970284d-b3d3-4186-9f94-4334aa2cb03b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:39.122249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 127/261] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:45:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:39.124000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970284d-b3d3-4186-9f94-4334aa2cb03b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:39.124867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091586c-5bf7-4636-9931-6c7a3fd9509b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:39.128233 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:39.161821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4c616-01d4-404f-921c-8c21dad9b553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:39.162868 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4c616-01d4-404f-921c-8c21dad9b553 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:39.163384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4c616-01d4-404f-921c-8c21dad9b553 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:39.163857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4c616-01d4-404f-921c-8c21dad9b553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:39.235554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4c616-01d4-404f-921c-8c21dad9b553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:39.237369 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4c616-01d4-404f-921c-8c21dad9b553 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:39.298935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970284d-b3d3-4186-9f94-4334aa2cb03b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:39.343554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 135/262] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:39.344779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4c616-01d4-404f-921c-8c21dad9b553 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:39.354739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:39.357556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a50ad-0831-4a94-a156-40b35c524263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:39.358336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a50ad-0831-4a94-a156-40b35c524263 None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:39.358675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a50ad-0831-4a94-a156-40b35c524263 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:39.359055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a50ad-0831-4a94-a156-40b35c524263 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46/users/f1acc21bb0b344db8bf6a6d5ce64b724/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:39.362180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a50ad-0831-4a94-a156-40b35c524263 None admin] RBAC: Authorizing `identity:create_grant(project_id=197b0ba2357a405ca5849973bb209b46, user_id=f1acc21bb0b344db8bf6a6d5ce64b724, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:39.370985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a50ad-0831-4a94-a156-40b35c524263 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:39.399163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 128/263] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:45:39 2022] PUT /identity/v3/projects/197b0ba2357a405ca5849973bb209b46/users/f1acc21bb0b344db8bf6a6d5ce64b724/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:39.400044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a50ad-0831-4a94-a156-40b35c524263 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:39.460679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:39.476295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4c616-01d4-404f-921c-8c21dad9b553 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:39.515827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60f710-02a7-44d7-8aea-5572cd2eb268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:39.517309 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60f710-02a7-44d7-8aea-5572cd2eb268 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:39.517847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60f710-02a7-44d7-8aea-5572cd2eb268 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:39.518338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60f710-02a7-44d7-8aea-5572cd2eb268 None admin] PATH_INFO: `/v3/users/cd13ff46a6e14dc68ee9f695b590cf66` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:39.520074 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60f710-02a7-44d7-8aea-5572cd2eb268 None admin] RBAC: Authorizing `identity:get_user(user_id=cd13ff46a6e14dc68ee9f695b590cf66)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:39.521861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60f710-02a7-44d7-8aea-5572cd2eb268 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:39.523893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 136/264] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:39 2022] GET /identity/v3/users/cd13ff46a6e14dc68ee9f695b590cf66 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:39.539200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60f710-02a7-44d7-8aea-5572cd2eb268 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:39.603073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a50ad-0831-4a94-a156-40b35c524263 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:39.666324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae73fa1-3fad-4728-8e72-15dbf1603976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:39.666324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae73fa1-3fad-4728-8e72-15dbf1603976 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:39.666324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae73fa1-3fad-4728-8e72-15dbf1603976 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:39.666324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae73fa1-3fad-4728-8e72-15dbf1603976 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:39.668219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae73fa1-3fad-4728-8e72-15dbf1603976 None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:39.669740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae73fa1-3fad-4728-8e72-15dbf1603976 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:39.672017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 129/265] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:39 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:39.674679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:39.679403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae73fa1-3fad-4728-8e72-15dbf1603976 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:39.726687 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60f710-02a7-44d7-8aea-5572cd2eb268 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:39.796644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed84d9a-6ed3-4924-843d-2759fd72c732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:39.797237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84d9a-6ed3-4924-843d-2759fd72c732 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:39.797525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84d9a-6ed3-4924-843d-2759fd72c732 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:39.797806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84d9a-6ed3-4924-843d-2759fd72c732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:39.848549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:39.862447 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae73fa1-3fad-4728-8e72-15dbf1603976 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:39.937392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27863c63-5ca9-4ef3-90ab-b0a3aeba2561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:39.938058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27863c63-5ca9-4ef3-90ab-b0a3aeba2561 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:39.938369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27863c63-5ca9-4ef3-90ab-b0a3aeba2561 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:39.938651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27863c63-5ca9-4ef3-90ab-b0a3aeba2561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:39.939559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27863c63-5ca9-4ef3-90ab-b0a3aeba2561 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=197b0ba2357a405ca5849973bb209b46, user.id=cd13ff46a6e14dc68ee9f695b590cf66) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:39.939872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27863c63-5ca9-4ef3-90ab-b0a3aeba2561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:39.941321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27863c63-5ca9-4ef3-90ab-b0a3aeba2561 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:39.946570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed84d9a-6ed3-4924-843d-2759fd72c732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:39.948028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed84d9a-6ed3-4924-843d-2759fd72c732 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:39.959276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 130/266] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:39 2022] GET /identity/v3/role_assignments?user.id=cd13ff46a6e14dc68ee9f695b590cf66&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=0c473678b281495f86f58b077be04e1b => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:39.960729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3ca42-bdaa-4402-a346-42f08f97c272 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:39.961017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3ca42-bdaa-4402-a346-42f08f97c272 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:39.961291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3ca42-bdaa-4402-a346-42f08f97c272 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:39.962707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 131/267] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:39.968587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e945f-a6c5-4493-aaa9-a2ece7fcf4fd None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:39.968891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e945f-a6c5-4493-aaa9-a2ece7fcf4fd None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:39.969152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e945f-a6c5-4493-aaa9-a2ece7fcf4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:40.007743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-fc9e945f-a6c5-4493-aaa9-a2ece7fcf4fd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:40.008962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-fc9e945f-a6c5-4493-aaa9-a2ece7fcf4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:40.099778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:40.121827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 132/268] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:40.158640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e945f-a6c5-4493-aaa9-a2ece7fcf4fd None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:40.165733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 137/269] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:45:40.171456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed84d9a-6ed3-4924-843d-2759fd72c732 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:40.278977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:40.297282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e945f-a6c5-4493-aaa9-a2ece7fcf4fd None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:40.336397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a9673-c323-458a-b555-9ee5eff2251a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:40.337192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a9673-c323-458a-b555-9ee5eff2251a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:40.337475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a9673-c323-458a-b555-9ee5eff2251a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:40.337747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a9673-c323-458a-b555-9ee5eff2251a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:40.350019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a9673-c323-458a-b555-9ee5eff2251a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-565a9673-c323-458a-b555-9ee5eff2251a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:40.361867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:40.362997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:40.365843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 133/270] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:40.371296 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a9673-c323-458a-b555-9ee5eff2251a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:40.456976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed84d9a-6ed3-4924-843d-2759fd72c732 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:40.480808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a9673-c323-458a-b555-9ee5eff2251a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:40.518933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919c24f-9e21-433b-b208-a6f185768b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:40.519900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919c24f-9e21-433b-b208-a6f185768b99 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:40.520183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919c24f-9e21-433b-b208-a6f185768b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:40.520615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919c24f-9e21-433b-b208-a6f185768b99 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:40.521379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919c24f-9e21-433b-b208-a6f185768b99 None admin] RBAC: Adding query filter params (name=member) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:40.521684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919c24f-9e21-433b-b208-a6f185768b99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:40.523466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919c24f-9e21-433b-b208-a6f185768b99 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:40.533856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 134/271] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:40.535767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbd8c9-da15-472a-85f0-fb714ed60910 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:40.535767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbd8c9-da15-472a-85f0-fb714ed60910 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:40.536085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbd8c9-da15-472a-85f0-fb714ed60910 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:40.537067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 135/272] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:40.538476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e59e7-4a6a-478f-967e-d0caadbe21d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:40.539925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e59e7-4a6a-478f-967e-d0caadbe21d8 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:40.540410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e59e7-4a6a-478f-967e-d0caadbe21d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:40.540892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e59e7-4a6a-478f-967e-d0caadbe21d8 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:40.542261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbd8c9-da15-472a-85f0-fb714ed60910 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:40.542593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e59e7-4a6a-478f-967e-d0caadbe21d8 None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:40.544253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e59e7-4a6a-478f-967e-d0caadbe21d8 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:40.547977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 138/273] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:40 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:40.549916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18f6a3-28ca-40e0-b70f-3a1b36c5a440 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:40.554824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18f6a3-28ca-40e0-b70f-3a1b36c5a440 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:40.554975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18f6a3-28ca-40e0-b70f-3a1b36c5a440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:40.563219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:40.580467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-4d18f6a3-28ca-40e0-b70f-3a1b36c5a440 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:40.582548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4d18f6a3-28ca-40e0-b70f-3a1b36c5a440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:40.634715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:40.671662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbd8c9-da15-472a-85f0-fb714ed60910 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:40.680345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 139/274] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:40.681443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18f6a3-28ca-40e0-b70f-3a1b36c5a440 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:40.726067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ddd1a-9a57-4521-8285-9adc529751f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:40.726994 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ddd1a-9a57-4521-8285-9adc529751f9 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:40.727279 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ddd1a-9a57-4521-8285-9adc529751f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:40.727675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ddd1a-9a57-4521-8285-9adc529751f9 None admin] PATH_INFO: `/v3/users/6e1f32c547fc42dbba63acee300255d6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:40.745068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ddd1a-9a57-4521-8285-9adc529751f9 None admin] RBAC: Authorizing `identity:get_user(user_id=6e1f32c547fc42dbba63acee300255d6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:40.746677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ddd1a-9a57-4521-8285-9adc529751f9 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:40.748779 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 136/275] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:40 2022] GET /identity/v3/users/6e1f32c547fc42dbba63acee300255d6 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:40.749685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ddd1a-9a57-4521-8285-9adc529751f9 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:40.817764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18f6a3-28ca-40e0-b70f-3a1b36c5a440 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:40.841721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:40.856791 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a69b5-3821-49e8-be3b-93f3790e7be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:40.857511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a69b5-3821-49e8-be3b-93f3790e7be2 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:40.857796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a69b5-3821-49e8-be3b-93f3790e7be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:40.858071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a69b5-3821-49e8-be3b-93f3790e7be2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:40.859012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a69b5-3821-49e8-be3b-93f3790e7be2 None admin] RBAC: Adding query filter params (role.id=c7b7d3b4dd944936851dee41ead5dc99, scope.project.id=197b0ba2357a405ca5849973bb209b46, user.id=cd13ff46a6e14dc68ee9f695b590cf66) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:40.859323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a69b5-3821-49e8-be3b-93f3790e7be2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:40.861534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a69b5-3821-49e8-be3b-93f3790e7be2 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:40.874311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 140/276] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:40 2022] GET /identity/v3/role_assignments?user.id=cd13ff46a6e14dc68ee9f695b590cf66&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=c7b7d3b4dd944936851dee41ead5dc99 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:40.874311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a69b5-3821-49e8-be3b-93f3790e7be2 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:40.904859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ddd1a-9a57-4521-8285-9adc529751f9 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:40.954896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:40.964749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a5e48-aa4c-4bf7-9874-4a11d109df14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:40.965558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a5e48-aa4c-4bf7-9874-4a11d109df14 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:40.965834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a5e48-aa4c-4bf7-9874-4a11d109df14 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:40.966152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a5e48-aa4c-4bf7-9874-4a11d109df14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:40.967559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a5e48-aa4c-4bf7-9874-4a11d109df14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:40.969054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a5e48-aa4c-4bf7-9874-4a11d109df14 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:40.971509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 137/277] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:40.972802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af56ec-d249-45f6-8385-833584d15c30 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:40.973081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af56ec-d249-45f6-8385-833584d15c30 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:40.973349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af56ec-d249-45f6-8385-833584d15c30 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:40.974830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 138/278] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:40.979736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af56ec-d249-45f6-8385-833584d15c30 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:41.030318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a69b5-3821-49e8-be3b-93f3790e7be2 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:41.071151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d081e0-8304-4d7e-a917-dde1bd940e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:41.071151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d081e0-8304-4d7e-a917-dde1bd940e79 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:41.071422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d081e0-8304-4d7e-a917-dde1bd940e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:41.071673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d081e0-8304-4d7e-a917-dde1bd940e79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:41.073153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d081e0-8304-4d7e-a917-dde1bd940e79 None admin] RBAC: Adding query filter params (role.id=27d8d0ea0b83488a95a9141d439247f6, user.id=6e1f32c547fc42dbba63acee300255d6, scope.system=all) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:41.073476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d081e0-8304-4d7e-a917-dde1bd940e79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:41.074964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d081e0-8304-4d7e-a917-dde1bd940e79 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:41.082536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 141/279] 10.222.0.45 () {56 vars in 1308 bytes} [Tue May 24 12:45:40 2022] GET /identity/v3/role_assignments?user.id=6e1f32c547fc42dbba63acee300255d6&scope.system=all&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:41.083978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a9814-7084-4d45-b7d8-ac6d4e3e1a89 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:41.084261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a9814-7084-4d45-b7d8-ac6d4e3e1a89 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:41.084542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a9814-7084-4d45-b7d8-ac6d4e3e1a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:41.100058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af56ec-d249-45f6-8385-833584d15c30 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:41.106318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-210a9814-7084-4d45-b7d8-ac6d4e3e1a89 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:41.107565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-210a9814-7084-4d45-b7d8-ac6d4e3e1a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:41.147668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2738533-5525-4b6c-a85f-970624f55c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:41.148511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2738533-5525-4b6c-a85f-970624f55c82 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:41.148791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2738533-5525-4b6c-a85f-970624f55c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:41.149072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2738533-5525-4b6c-a85f-970624f55c82 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:41.149925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2738533-5525-4b6c-a85f-970624f55c82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:41.151576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2738533-5525-4b6c-a85f-970624f55c82 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:41.165726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:41.184673 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 139/280] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:45:40 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:41.216255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 142/281] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:41.240379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2738533-5525-4b6c-a85f-970624f55c82 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:41.358731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2738533-5525-4b6c-a85f-970624f55c82 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:41.397701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2aa20-17a2-485f-9163-b4bfd889a669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:41.398545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2aa20-17a2-485f-9163-b4bfd889a669 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:41.398886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2aa20-17a2-485f-9163-b4bfd889a669 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:41.399160 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2aa20-17a2-485f-9163-b4bfd889a669 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:41.411622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2aa20-17a2-485f-9163-b4bfd889a669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-aaf2aa20-17a2-485f-9163-b4bfd889a669 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:41.419962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:41.421087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:41.422052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:41.424849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 140/282] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:41.430634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a9814-7084-4d45-b7d8-ac6d4e3e1a89 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:41.459646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:41.536283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a9814-7084-4d45-b7d8-ac6d4e3e1a89 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:41.573307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6335d-6330-4648-ba1f-3bc4452a03ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:41.574065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6335d-6330-4648-ba1f-3bc4452a03ba None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:41.574264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6335d-6330-4648-ba1f-3bc4452a03ba None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:41.574620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6335d-6330-4648-ba1f-3bc4452a03ba None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:41.575372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6335d-6330-4648-ba1f-3bc4452a03ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:41.575726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6335d-6330-4648-ba1f-3bc4452a03ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:41.577096 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6335d-6330-4648-ba1f-3bc4452a03ba None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:41.586562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 143/283] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:41.592071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2aa20-17a2-485f-9163-b4bfd889a669 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:41.653690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:41.702929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2aa20-17a2-485f-9163-b4bfd889a669 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:41.736976 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea58fb-d303-4c54-a32e-0c4c4dea7693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:41.737857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea58fb-d303-4c54-a32e-0c4c4dea7693 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:41.738142 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea58fb-d303-4c54-a32e-0c4c4dea7693 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:41.738482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea58fb-d303-4c54-a32e-0c4c4dea7693 None admin] PATH_INFO: `/v3/users/67a76e9e72e045769a678e7b96f8dbf3` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:41.739690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea58fb-d303-4c54-a32e-0c4c4dea7693 None admin] RBAC: Authorizing `identity:get_user(user_id=67a76e9e72e045769a678e7b96f8dbf3)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:41.741156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea58fb-d303-4c54-a32e-0c4c4dea7693 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:41.743120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 141/284] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:41 2022] GET /identity/v3/users/67a76e9e72e045769a678e7b96f8dbf3 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:41.750985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6335d-6330-4648-ba1f-3bc4452a03ba None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:41.885802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:41.902186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6335d-6330-4648-ba1f-3bc4452a03ba None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:41.953458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8a551-3b6c-4201-b27a-76c486ace28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:41.954631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8a551-3b6c-4201-b27a-76c486ace28c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:41.955078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8a551-3b6c-4201-b27a-76c486ace28c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:41.955495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8a551-3b6c-4201-b27a-76c486ace28c None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:41.956847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8a551-3b6c-4201-b27a-76c486ace28c None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:41.958470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8a551-3b6c-4201-b27a-76c486ace28c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:41.961258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 144/285] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:41 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:41.965904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea58fb-d303-4c54-a32e-0c4c4dea7693 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:42.107115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea58fb-d303-4c54-a32e-0c4c4dea7693 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:42.115018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c7dd7-06bb-415c-af50-8c36859af23a None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:42.115583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c7dd7-06bb-415c-af50-8c36859af23a None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:42.116070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c7dd7-06bb-415c-af50-8c36859af23a None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:42.117645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 145/286] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:45:42.122844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0978d5f-9f67-4ac5-9ca7-b1c70520dd13 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:42.123202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0978d5f-9f67-4ac5-9ca7-b1c70520dd13 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:42.123576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0978d5f-9f67-4ac5-9ca7-b1c70520dd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:42.142633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:42.154939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-c0978d5f-9f67-4ac5-9ca7-b1c70520dd13 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:42.158704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c0978d5f-9f67-4ac5-9ca7-b1c70520dd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:42.162147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ece96b-8bde-4686-8fcb-c5b185970674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:42.163286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ece96b-8bde-4686-8fcb-c5b185970674 None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:42.163748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ece96b-8bde-4686-8fcb-c5b185970674 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:42.164217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ece96b-8bde-4686-8fcb-c5b185970674 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46/users/67a76e9e72e045769a678e7b96f8dbf3/roles/4a71e72752894223a2b3fd53f728c211` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:42.166451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ece96b-8bde-4686-8fcb-c5b185970674 None admin] RBAC: Authorizing `identity:create_grant(project_id=197b0ba2357a405ca5849973bb209b46, user_id=67a76e9e72e045769a678e7b96f8dbf3, role_id=4a71e72752894223a2b3fd53f728c211)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:42.168184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ece96b-8bde-4686-8fcb-c5b185970674 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:42.192222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 142/287] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:45:41 2022] PUT /identity/v3/projects/197b0ba2357a405ca5849973bb209b46/users/67a76e9e72e045769a678e7b96f8dbf3/roles/4a71e72752894223a2b3fd53f728c211 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:42.263317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 146/288] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:42.303137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ece96b-8bde-4686-8fcb-c5b185970674 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:42.448105 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ece96b-8bde-4686-8fcb-c5b185970674 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:42.500183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fac776-9975-47bb-8183-e0c6d13d0a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:42.500987 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fac776-9975-47bb-8183-e0c6d13d0a9c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:42.501798 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fac776-9975-47bb-8183-e0c6d13d0a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:42.502074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fac776-9975-47bb-8183-e0c6d13d0a9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:42.514884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fac776-9975-47bb-8183-e0c6d13d0a9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-89fac776-9975-47bb-8183-e0c6d13d0a9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:42.522644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:42.523568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:42.524367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 143/289] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:42.529773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0978d5f-9f67-4ac5-9ca7-b1c70520dd13 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:42.618102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:42.630971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0978d5f-9f67-4ac5-9ca7-b1c70520dd13 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:42.666235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877319-e3ed-40c5-81ee-3095b41cb450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:42.666934 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877319-e3ed-40c5-81ee-3095b41cb450 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:42.667217 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877319-e3ed-40c5-81ee-3095b41cb450 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:42.667493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877319-e3ed-40c5-81ee-3095b41cb450 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:42.668247 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877319-e3ed-40c5-81ee-3095b41cb450 None admin] RBAC: Adding query filter params (name=member) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:42.668550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877319-e3ed-40c5-81ee-3095b41cb450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:42.669930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877319-e3ed-40c5-81ee-3095b41cb450 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:42.680052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 147/290] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:42.685637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fac776-9975-47bb-8183-e0c6d13d0a9c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:42.744033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:42.820945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fac776-9975-47bb-8183-e0c6d13d0a9c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:42.856646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8a1eb-fdec-4817-b659-e84046ae151b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:42.857527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a1eb-fdec-4817-b659-e84046ae151b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:42.857804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a1eb-fdec-4817-b659-e84046ae151b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:42.858080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a1eb-fdec-4817-b659-e84046ae151b None admin] PATH_INFO: `/v3/users/f1acc21bb0b344db8bf6a6d5ce64b724` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:42.859299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8a1eb-fdec-4817-b659-e84046ae151b None admin] RBAC: Authorizing `identity:get_user(user_id=f1acc21bb0b344db8bf6a6d5ce64b724)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:42.861272 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8a1eb-fdec-4817-b659-e84046ae151b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:42.862931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 144/291] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:42 2022] GET /identity/v3/users/f1acc21bb0b344db8bf6a6d5ce64b724 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:42.868512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877319-e3ed-40c5-81ee-3095b41cb450 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:42.969635 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877319-e3ed-40c5-81ee-3095b41cb450 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:43.004374 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f9be-fa25-4565-80f1-cd85f9b0df4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:43.005027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f9be-fa25-4565-80f1-cd85f9b0df4a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:43.005311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f9be-fa25-4565-80f1-cd85f9b0df4a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:43.005588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f9be-fa25-4565-80f1-cd85f9b0df4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:43.025751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:43.087692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f9be-fa25-4565-80f1-cd85f9b0df4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:43.089173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f9be-fa25-4565-80f1-cd85f9b0df4a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:43.201470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 148/292] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:43.207804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8a1eb-fdec-4817-b659-e84046ae151b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:43.307314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8a1eb-fdec-4817-b659-e84046ae151b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:43.318238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:43.341831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02709a3-543f-4364-9c43-aa5347ae9c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:43.342694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02709a3-543f-4364-9c43-aa5347ae9c1e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:43.342967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02709a3-543f-4364-9c43-aa5347ae9c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:43.343246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02709a3-543f-4364-9c43-aa5347ae9c1e None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:43.344485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02709a3-543f-4364-9c43-aa5347ae9c1e None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:43.345964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02709a3-543f-4364-9c43-aa5347ae9c1e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:43.351263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 145/293] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:43 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:43.353903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f9be-fa25-4565-80f1-cd85f9b0df4a None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:43.436618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:43.476126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f9be-fa25-4565-80f1-cd85f9b0df4a None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:43.509851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd34d7f-333a-4f0c-aa35-7d16ebd5f0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:43.510537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd34d7f-333a-4f0c-aa35-7d16ebd5f0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:43.510818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd34d7f-333a-4f0c-aa35-7d16ebd5f0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:43.511089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd34d7f-333a-4f0c-aa35-7d16ebd5f0b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:43.512173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd34d7f-333a-4f0c-aa35-7d16ebd5f0b3 None admin] RBAC: Adding query filter params (role.id=27d8d0ea0b83488a95a9141d439247f6, scope.project.id=197b0ba2357a405ca5849973bb209b46, user.id=f1acc21bb0b344db8bf6a6d5ce64b724) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:43.512650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd34d7f-333a-4f0c-aa35-7d16ebd5f0b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:43.514245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd34d7f-333a-4f0c-aa35-7d16ebd5f0b3 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:43.520779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 149/294] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:43 2022] GET /identity/v3/role_assignments?user.id=f1acc21bb0b344db8bf6a6d5ce64b724&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:44.002576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec869b02-43f0-451d-a1dd-bdf6c31740cc None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.003218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec869b02-43f0-451d-a1dd-bdf6c31740cc None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.003218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec869b02-43f0-451d-a1dd-bdf6c31740cc None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.004828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 146/295] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:44.010344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa82baa-dca1-4a2b-8326-9b5b6142bbd3 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.010864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa82baa-dca1-4a2b-8326-9b5b6142bbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.011017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa82baa-dca1-4a2b-8326-9b5b6142bbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.029543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7aa82baa-dca1-4a2b-8326-9b5b6142bbd3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:44.030762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7aa82baa-dca1-4a2b-8326-9b5b6142bbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:44.102730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 150/296] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:44.125793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec869b02-43f0-451d-a1dd-bdf6c31740cc None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:44.238093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec869b02-43f0-451d-a1dd-bdf6c31740cc None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:44.272250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63095f09-8762-4305-a32a-427eba6d25b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:44.272968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63095f09-8762-4305-a32a-427eba6d25b3 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.273248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63095f09-8762-4305-a32a-427eba6d25b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.273526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63095f09-8762-4305-a32a-427eba6d25b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.284356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63095f09-8762-4305-a32a-427eba6d25b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-63095f09-8762-4305-a32a-427eba6d25b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:44.292361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:44.293252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:44.294031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:44.298565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 147/297] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:44.300264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa82baa-dca1-4a2b-8326-9b5b6142bbd3 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:44.327335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8e18f-4ad7-4963-85cb-17413143753a None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.327649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8e18f-4ad7-4963-85cb-17413143753a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.327902 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8e18f-4ad7-4963-85cb-17413143753a None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.329772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 148/298] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:45:44.334857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00493a99-b3dd-4d47-aaf5-8f80def71851 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.335148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00493a99-b3dd-4d47-aaf5-8f80def71851 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.335430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00493a99-b3dd-4d47-aaf5-8f80def71851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.349270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:44.353920 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-00493a99-b3dd-4d47-aaf5-8f80def71851 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:44.355157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-00493a99-b3dd-4d47-aaf5-8f80def71851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:44.390536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:44.427620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa82baa-dca1-4a2b-8326-9b5b6142bbd3 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:44.438322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 149/299] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:44.440121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e8680-a0ed-4aa0-aeab-72e5756cac33 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.440412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e8680-a0ed-4aa0-aeab-72e5756cac33 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.440688 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e8680-a0ed-4aa0-aeab-72e5756cac33 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.442074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 150/300] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:44.447539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4edb3e-8b4f-4dc0-8f13-88812b59bc0f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.447831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4edb3e-8b4f-4dc0-8f13-88812b59bc0f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.448116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4edb3e-8b4f-4dc0-8f13-88812b59bc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.466022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8a4edb3e-8b4f-4dc0-8f13-88812b59bc0f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:44.467244 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8a4edb3e-8b4f-4dc0-8f13-88812b59bc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:44.481303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3caa7d-bb1b-4e9d-91ad-7fffdd34f834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:44.481984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3caa7d-bb1b-4e9d-91ad-7fffdd34f834 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.482277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3caa7d-bb1b-4e9d-91ad-7fffdd34f834 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.482580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3caa7d-bb1b-4e9d-91ad-7fffdd34f834 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.483345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3caa7d-bb1b-4e9d-91ad-7fffdd34f834 None admin] RBAC: Adding query filter params (name=member) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:44.483644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3caa7d-bb1b-4e9d-91ad-7fffdd34f834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:44.485031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3caa7d-bb1b-4e9d-91ad-7fffdd34f834 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:44.498654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 151/301] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:44.499528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3caa7d-bb1b-4e9d-91ad-7fffdd34f834 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:44.546509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 151/302] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:44.547624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4edb3e-8b4f-4dc0-8f13-88812b59bc0f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:44.620763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3caa7d-bb1b-4e9d-91ad-7fffdd34f834 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:44.628763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:44.655413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:44.658350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfc717-7748-4c5c-91c3-a67c23762b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:44.659355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc717-7748-4c5c-91c3-a67c23762b0e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.659635 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc717-7748-4c5c-91c3-a67c23762b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.659905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc717-7748-4c5c-91c3-a67c23762b0e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.660114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4edb3e-8b4f-4dc0-8f13-88812b59bc0f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:44.670339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfc717-7748-4c5c-91c3-a67c23762b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-b4cfc717-7748-4c5c-91c3-a67c23762b0e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:44.678136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:44.679162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:44.680575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 152/303] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:44.684647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfc717-7748-4c5c-91c3-a67c23762b0e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:44.698302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125faf21-bdaf-4499-95b6-7256445b0cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:44.699183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125faf21-bdaf-4499-95b6-7256445b0cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.699469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125faf21-bdaf-4499-95b6-7256445b0cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.699745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125faf21-bdaf-4499-95b6-7256445b0cb6 None admin] PATH_INFO: `/v3/users/6e1f32c547fc42dbba63acee300255d6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.700934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125faf21-bdaf-4499-95b6-7256445b0cb6 None admin] RBAC: Authorizing `identity:get_user(user_id=6e1f32c547fc42dbba63acee300255d6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:44.702463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125faf21-bdaf-4499-95b6-7256445b0cb6 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:44.704228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 152/304] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:44 2022] GET /identity/v3/users/6e1f32c547fc42dbba63acee300255d6 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:44.709387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125faf21-bdaf-4499-95b6-7256445b0cb6 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:44.787068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfc717-7748-4c5c-91c3-a67c23762b0e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:44.815177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125faf21-bdaf-4499-95b6-7256445b0cb6 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:44.821304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756f5d6-a52c-483c-a2c7-f4220ba2bd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:44.822093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756f5d6-a52c-483c-a2c7-f4220ba2bd99 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.822511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756f5d6-a52c-483c-a2c7-f4220ba2bd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.822794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756f5d6-a52c-483c-a2c7-f4220ba2bd99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.823944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756f5d6-a52c-483c-a2c7-f4220ba2bd99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:44.825372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756f5d6-a52c-483c-a2c7-f4220ba2bd99 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:44.826994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 153/305] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:44.827716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756f5d6-a52c-483c-a2c7-f4220ba2bd99 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:44.849549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8a7ab-eabb-4648-903f-a0564b919bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:44.850218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8a7ab-eabb-4648-903f-a0564b919bbf None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.850536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8a7ab-eabb-4648-903f-a0564b919bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.850814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8a7ab-eabb-4648-903f-a0564b919bbf None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.851562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8a7ab-eabb-4648-903f-a0564b919bbf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:44.851870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8a7ab-eabb-4648-903f-a0564b919bbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:44.853245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8a7ab-eabb-4648-903f-a0564b919bbf None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:44.861591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 153/306] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:44.862808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902084e7-3ee6-4fd4-9cf3-da319b1f4b4f None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.863097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902084e7-3ee6-4fd4-9cf3-da319b1f4b4f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.863374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902084e7-3ee6-4fd4-9cf3-da319b1f4b4f None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.864600 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 154/307] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:44.865686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902084e7-3ee6-4fd4-9cf3-da319b1f4b4f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:44.866716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:44.893010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:44.935738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756f5d6-a52c-483c-a2c7-f4220ba2bd99 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:44.965818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902084e7-3ee6-4fd4-9cf3-da319b1f4b4f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:44.968221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba53646-f01c-46b5-9b88-4955867499c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:44.969085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba53646-f01c-46b5-9b88-4955867499c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:44.969368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba53646-f01c-46b5-9b88-4955867499c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:44.969647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba53646-f01c-46b5-9b88-4955867499c0 None admin] PATH_INFO: `/v3/system/users/6e1f32c547fc42dbba63acee300255d6/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:44.971202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba53646-f01c-46b5-9b88-4955867499c0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6e1f32c547fc42dbba63acee300255d6, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:44.972579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba53646-f01c-46b5-9b88-4955867499c0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:44.981082 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 154/308] 10.222.0.45 () {58 vars in 1334 bytes} [Tue May 24 12:45:44 2022] PUT /identity/v3/system/users/6e1f32c547fc42dbba63acee300255d6/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:44.982061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba53646-f01c-46b5-9b88-4955867499c0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:45.008471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812039a-7fc9-4add-a469-271221915fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:45.009228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812039a-7fc9-4add-a469-271221915fc8 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:45.009498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812039a-7fc9-4add-a469-271221915fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:45.009818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812039a-7fc9-4add-a469-271221915fc8 None admin] PATH_INFO: `/v3/groups` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:45.010573 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812039a-7fc9-4add-a469-271221915fc8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:45.013104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812039a-7fc9-4add-a469-271221915fc8 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:45.029715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 155/309] 10.222.0.45 () {60 vars in 1134 bytes} [Tue May 24 12:45:44 2022] POST /identity/v3/groups => generated 229 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:45:45.030843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162794f-e7e4-4da1-ac9c-c8aeebfccefc None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:45.031132 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162794f-e7e4-4da1-ac9c-c8aeebfccefc None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:45.031463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162794f-e7e4-4da1-ac9c-c8aeebfccefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:45.050720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-4162794f-e7e4-4da1-ac9c-c8aeebfccefc None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:45.051968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4162794f-e7e4-4da1-ac9c-c8aeebfccefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:45.114106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:45.121761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba53646-f01c-46b5-9b88-4955867499c0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:45.133799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 156/310] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:45.158494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162794f-e7e4-4da1-ac9c-c8aeebfccefc None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:45.167598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312da23-72bb-44b4-910e-e5d64bfb97a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:45.168884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312da23-72bb-44b4-910e-e5d64bfb97a0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:45.169472 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312da23-72bb-44b4-910e-e5d64bfb97a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:45.170124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312da23-72bb-44b4-910e-e5d64bfb97a0 None admin] PATH_INFO: `/v3/users/8a74a957f39f475c8a19f8f364c46132` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:45.183047 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:45.189144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312da23-72bb-44b4-910e-e5d64bfb97a0 None admin] RBAC: Authorizing `identity:get_user(user_id=8a74a957f39f475c8a19f8f364c46132)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:45.191042 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312da23-72bb-44b4-910e-e5d64bfb97a0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:45.192923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 155/311] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:44 2022] GET /identity/v3/users/8a74a957f39f475c8a19f8f364c46132 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:45.199325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312da23-72bb-44b4-910e-e5d64bfb97a0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:45.289104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162794f-e7e4-4da1-ac9c-c8aeebfccefc None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:45.331784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31f9fa-39cc-477b-9ea1-ba46df777c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:45.332983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31f9fa-39cc-477b-9ea1-ba46df777c9b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:45.333363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31f9fa-39cc-477b-9ea1-ba46df777c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:45.333735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31f9fa-39cc-477b-9ea1-ba46df777c9b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:45.334808 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312da23-72bb-44b4-910e-e5d64bfb97a0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:45.349878 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31f9fa-39cc-477b-9ea1-ba46df777c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-5e31f9fa-39cc-477b-9ea1-ba46df777c9b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:45.360159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:45.361219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:45.362018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 157/312] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:45.367133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31f9fa-39cc-477b-9ea1-ba46df777c9b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:45.378873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a5d28-c940-4dd5-b94f-ae3547dc7b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:45.379772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a5d28-c940-4dd5-b94f-ae3547dc7b43 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:45.379882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a5d28-c940-4dd5-b94f-ae3547dc7b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:45.380123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a5d28-c940-4dd5-b94f-ae3547dc7b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:45.380961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a5d28-c940-4dd5-b94f-ae3547dc7b43 None admin] RBAC: Adding query filter params (role.id=4a71e72752894223a2b3fd53f728c211, user.id=8a74a957f39f475c8a19f8f364c46132, scope.system=all) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:45.381272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a5d28-c940-4dd5-b94f-ae3547dc7b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:45.383277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a5d28-c940-4dd5-b94f-ae3547dc7b43 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:45.398461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 156/313] 10.222.0.45 () {56 vars in 1308 bytes} [Tue May 24 12:45:45 2022] GET /identity/v3/role_assignments?user.id=8a74a957f39f475c8a19f8f364c46132&scope.system=all&role.id=4a71e72752894223a2b3fd53f728c211 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:45.463763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:45.490450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31f9fa-39cc-477b-9ea1-ba46df777c9b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:45.529222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539af20-31d1-4b9b-ab15-9ae0d7cbe573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:45.530312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539af20-31d1-4b9b-ab15-9ae0d7cbe573 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:45.530807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539af20-31d1-4b9b-ab15-9ae0d7cbe573 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:45.531307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539af20-31d1-4b9b-ab15-9ae0d7cbe573 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:45.532826 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539af20-31d1-4b9b-ab15-9ae0d7cbe573 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:45.533371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539af20-31d1-4b9b-ab15-9ae0d7cbe573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:45.535032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539af20-31d1-4b9b-ab15-9ae0d7cbe573 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:45.544942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 158/314] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:45.550468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a5d28-c940-4dd5-b94f-ae3547dc7b43 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:45.580649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:45.666755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a5d28-c940-4dd5-b94f-ae3547dc7b43 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:45.702672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f83c23-a4f6-41f5-ab97-c1fee771221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:45.703557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f83c23-a4f6-41f5-ab97-c1fee771221f None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:45.703843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f83c23-a4f6-41f5-ab97-c1fee771221f None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:45.704149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f83c23-a4f6-41f5-ab97-c1fee771221f None admin] PATH_INFO: `/v3/users/67a76e9e72e045769a678e7b96f8dbf3` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:45.705864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f83c23-a4f6-41f5-ab97-c1fee771221f None admin] RBAC: Authorizing `identity:get_user(user_id=67a76e9e72e045769a678e7b96f8dbf3)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:45.707346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f83c23-a4f6-41f5-ab97-c1fee771221f None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:45.709296 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 157/315] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:45 2022] GET /identity/v3/users/67a76e9e72e045769a678e7b96f8dbf3 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:45.714402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539af20-31d1-4b9b-ab15-9ae0d7cbe573 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:45.825127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539af20-31d1-4b9b-ab15-9ae0d7cbe573 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:45.859656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521297aa-d737-4250-b059-4840605d921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:45.860320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521297aa-d737-4250-b059-4840605d921e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:45.861138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521297aa-d737-4250-b059-4840605d921e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:45.861416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521297aa-d737-4250-b059-4840605d921e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:45.872740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:45.934818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521297aa-d737-4250-b059-4840605d921e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:45.936228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521297aa-d737-4250-b059-4840605d921e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:46.041755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 159/316] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:46.047500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f83c23-a4f6-41f5-ab97-c1fee771221f None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:46.138248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:46.151308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f83c23-a4f6-41f5-ab97-c1fee771221f None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:46.190207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4982fbc-2b3e-4b30-8132-709cd97e2e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:46.191012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4982fbc-2b3e-4b30-8132-709cd97e2e14 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:46.191291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4982fbc-2b3e-4b30-8132-709cd97e2e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:46.191565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4982fbc-2b3e-4b30-8132-709cd97e2e14 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:46.192744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4982fbc-2b3e-4b30-8132-709cd97e2e14 None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:46.194171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4982fbc-2b3e-4b30-8132-709cd97e2e14 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:46.196188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 158/317] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:46 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:46.201879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521297aa-d737-4250-b059-4840605d921e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:46.245516 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:46.307880 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521297aa-d737-4250-b059-4840605d921e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:46.342862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45695ec3-199b-406a-b539-cccde7a1f11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:46.343527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695ec3-199b-406a-b539-cccde7a1f11c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:46.343809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695ec3-199b-406a-b539-cccde7a1f11c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:46.344083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695ec3-199b-406a-b539-cccde7a1f11c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:46.344947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45695ec3-199b-406a-b539-cccde7a1f11c None admin] RBAC: Adding query filter params (role.id=4a71e72752894223a2b3fd53f728c211, scope.project.id=197b0ba2357a405ca5849973bb209b46, user.id=67a76e9e72e045769a678e7b96f8dbf3) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:46.345260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45695ec3-199b-406a-b539-cccde7a1f11c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:46.346743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45695ec3-199b-406a-b539-cccde7a1f11c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:46.353071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 160/318] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:45:46 2022] GET /identity/v3/role_assignments?user.id=67a76e9e72e045769a678e7b96f8dbf3&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=4a71e72752894223a2b3fd53f728c211 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:47.271258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8aefe-9e65-4a25-a801-55847ec69ca1 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.271822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8aefe-9e65-4a25-a801-55847ec69ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.271822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8aefe-9e65-4a25-a801-55847ec69ca1 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.273431 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 159/319] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:47.278613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e762eb-ce79-4568-9e23-a01019558bfd None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.279065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e762eb-ce79-4568-9e23-a01019558bfd None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.279210 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e762eb-ce79-4568-9e23-a01019558bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.296338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a2e762eb-ce79-4568-9e23-a01019558bfd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:47.297501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a2e762eb-ce79-4568-9e23-a01019558bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:47.365307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 161/320] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:47.384341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863f4dc-f712-45e9-b26e-7291ff72686d None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.384341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863f4dc-f712-45e9-b26e-7291ff72686d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.384341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863f4dc-f712-45e9-b26e-7291ff72686d None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.386398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 160/321] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:47.390841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e762eb-ce79-4568-9e23-a01019558bfd None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:47.396484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932412ce-6d33-44b8-9852-ce27b62e210e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.396779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932412ce-6d33-44b8-9852-ce27b62e210e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.397057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932412ce-6d33-44b8-9852-ce27b62e210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.414394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-932412ce-6d33-44b8-9852-ce27b62e210e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:47.415594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-932412ce-6d33-44b8-9852-ce27b62e210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:47.426562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:47.494942 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 161/322] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:47.504161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e762eb-ce79-4568-9e23-a01019558bfd None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:47.517259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932412ce-6d33-44b8-9852-ce27b62e210e None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:47.542274 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfde651-e80c-4afc-85e1-d4cecdceac00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:47.543016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfde651-e80c-4afc-85e1-d4cecdceac00 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.543301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfde651-e80c-4afc-85e1-d4cecdceac00 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.543575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfde651-e80c-4afc-85e1-d4cecdceac00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.554391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfde651-e80c-4afc-85e1-d4cecdceac00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-bdfde651-e80c-4afc-85e1-d4cecdceac00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:47.562059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:47.563089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:47.564020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 162/323] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:47.568911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfde651-e80c-4afc-85e1-d4cecdceac00 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:47.607538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:47.623449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932412ce-6d33-44b8-9852-ce27b62e210e None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:47.657399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052c53d-68cd-486c-9945-861ba0ec8d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:47.657860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052c53d-68cd-486c-9945-861ba0ec8d5a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.658144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052c53d-68cd-486c-9945-861ba0ec8d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.658444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052c53d-68cd-486c-9945-861ba0ec8d5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.659605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052c53d-68cd-486c-9945-861ba0ec8d5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:47.661036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052c53d-68cd-486c-9945-861ba0ec8d5a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:47.662640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:47.662748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 162/324] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:45:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:47.663810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f546-e793-4239-a346-7719f440e533 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.664095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f546-e793-4239-a346-7719f440e533 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.664373 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f546-e793-4239-a346-7719f440e533 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.665817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 163/325] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:47.667964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f546-e793-4239-a346-7719f440e533 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:47.673288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfde651-e80c-4afc-85e1-d4cecdceac00 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:47.706623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481d15d-03f5-4d5e-a28d-442cad74a175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:47.707292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481d15d-03f5-4d5e-a28d-442cad74a175 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.707580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481d15d-03f5-4d5e-a28d-442cad74a175 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.707860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481d15d-03f5-4d5e-a28d-442cad74a175 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.708611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481d15d-03f5-4d5e-a28d-442cad74a175 None admin] RBAC: Adding query filter params (name=member) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:47.708927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481d15d-03f5-4d5e-a28d-442cad74a175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:47.710324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481d15d-03f5-4d5e-a28d-442cad74a175 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:47.718332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 163/326] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:47.718332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c0997-a975-4fbe-a70a-5c1545dab89f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.718332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c0997-a975-4fbe-a70a-5c1545dab89f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.718332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c0997-a975-4fbe-a70a-5c1545dab89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.735642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-cd8c0997-a975-4fbe-a70a-5c1545dab89f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:47.736806 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-cd8c0997-a975-4fbe-a70a-5c1545dab89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:47.768240 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f546-e793-4239-a346-7719f440e533 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:47.800937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef6f8-d993-4254-8e46-5dcac1ea57fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:47.801553 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef6f8-d993-4254-8e46-5dcac1ea57fe None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.801835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef6f8-d993-4254-8e46-5dcac1ea57fe None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.802114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef6f8-d993-4254-8e46-5dcac1ea57fe None admin] PATH_INFO: `/v3/groups` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.802826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ef6f8-d993-4254-8e46-5dcac1ea57fe None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:47.803713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 164/327] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:47.804263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ef6f8-d993-4254-8e46-5dcac1ea57fe None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:47.804545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c0997-a975-4fbe-a70a-5c1545dab89f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:47.830380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 164/328] 10.222.0.45 () {60 vars in 1134 bytes} [Tue May 24 12:45:47 2022] POST /identity/v3/groups => generated 226 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:45:47.830380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef6f8-d993-4254-8e46-5dcac1ea57fe None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:47.845530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:47.918915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:47.936146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c0997-a975-4fbe-a70a-5c1545dab89f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:47.942736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef6f8-d993-4254-8e46-5dcac1ea57fe None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:47.975797 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee6097-7f34-473e-9f02-e318f69f5282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:47.976685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee6097-7f34-473e-9f02-e318f69f5282 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.976967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee6097-7f34-473e-9f02-e318f69f5282 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.977238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee6097-7f34-473e-9f02-e318f69f5282 None admin] PATH_INFO: `/v3/users/6e1f32c547fc42dbba63acee300255d6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.978477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee6097-7f34-473e-9f02-e318f69f5282 None admin] RBAC: Authorizing `identity:get_user(user_id=6e1f32c547fc42dbba63acee300255d6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:47.979363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8984b8-e0b1-4b79-9c2f-8aa2d9cd8c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:47.979972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee6097-7f34-473e-9f02-e318f69f5282 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:47.980183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8984b8-e0b1-4b79-9c2f-8aa2d9cd8c91 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:47.980473 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8984b8-e0b1-4b79-9c2f-8aa2d9cd8c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:47.980755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8984b8-e0b1-4b79-9c2f-8aa2d9cd8c91 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:47.987178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 165/329] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:47 2022] GET /identity/v3/users/6e1f32c547fc42dbba63acee300255d6 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:47.988041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee6097-7f34-473e-9f02-e318f69f5282 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:47.994678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8984b8-e0b1-4b79-9c2f-8aa2d9cd8c91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-5d8984b8-e0b1-4b79-9c2f-8aa2d9cd8c91 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:48.004277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:48.005193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:48.005980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 165/330] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:48.013400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8984b8-e0b1-4b79-9c2f-8aa2d9cd8c91 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:48.108656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:48.119340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee6097-7f34-473e-9f02-e318f69f5282 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:48.138621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8984b8-e0b1-4b79-9c2f-8aa2d9cd8c91 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:48.164879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e80fd-1457-4a83-86c7-20b948a1ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:48.165962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e80fd-1457-4a83-86c7-20b948a1ccf7 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:48.166600 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e80fd-1457-4a83-86c7-20b948a1ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:48.167059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e80fd-1457-4a83-86c7-20b948a1ccf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:48.168782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e80fd-1457-4a83-86c7-20b948a1ccf7 None admin] RBAC: Adding query filter params (role.id=27d8d0ea0b83488a95a9141d439247f6, user.id=6e1f32c547fc42dbba63acee300255d6, scope.system=all) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:48.169288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e80fd-1457-4a83-86c7-20b948a1ccf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:48.170963 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e80fd-1457-4a83-86c7-20b948a1ccf7 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:48.184173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1153c-ed39-45a2-89fc-6989c934722a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:48.185172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1153c-ed39-45a2-89fc-6989c934722a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:48.185657 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1153c-ed39-45a2-89fc-6989c934722a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:48.185932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 166/331] 10.222.0.45 () {56 vars in 1308 bytes} [Tue May 24 12:45:47 2022] GET /identity/v3/role_assignments?user.id=6e1f32c547fc42dbba63acee300255d6&scope.system=all&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:48.186360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1153c-ed39-45a2-89fc-6989c934722a None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:48.187351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1153c-ed39-45a2-89fc-6989c934722a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:48.187795 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1153c-ed39-45a2-89fc-6989c934722a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:48.189846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1153c-ed39-45a2-89fc-6989c934722a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:48.200292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 166/332] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:48.206390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e80fd-1457-4a83-86c7-20b948a1ccf7 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:48.225904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:48.334624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e80fd-1457-4a83-86c7-20b948a1ccf7 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:48.375874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8401a4c-2bfb-4061-8447-dcf517eb55de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:48.377094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8401a4c-2bfb-4061-8447-dcf517eb55de None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:48.377558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8401a4c-2bfb-4061-8447-dcf517eb55de None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:48.378030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8401a4c-2bfb-4061-8447-dcf517eb55de None admin] PATH_INFO: `/v3/users/8a74a957f39f475c8a19f8f364c46132` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:48.379535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8401a4c-2bfb-4061-8447-dcf517eb55de None admin] RBAC: Authorizing `identity:get_user(user_id=8a74a957f39f475c8a19f8f364c46132)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:48.381180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8401a4c-2bfb-4061-8447-dcf517eb55de None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:48.383872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 167/333] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:48 2022] GET /identity/v3/users/8a74a957f39f475c8a19f8f364c46132 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:48.392129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1153c-ed39-45a2-89fc-6989c934722a None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:48.514655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1153c-ed39-45a2-89fc-6989c934722a None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:48.551362 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a17aea-b031-407e-9237-be686b1c707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:48.552234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a17aea-b031-407e-9237-be686b1c707b None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:48.552565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a17aea-b031-407e-9237-be686b1c707b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:48.552849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a17aea-b031-407e-9237-be686b1c707b None admin] PATH_INFO: `/v3/system/users/8a74a957f39f475c8a19f8f364c46132/roles/4a71e72752894223a2b3fd53f728c211` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:48.554389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a17aea-b031-407e-9237-be686b1c707b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8a74a957f39f475c8a19f8f364c46132, role_id=4a71e72752894223a2b3fd53f728c211)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:48.555782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a17aea-b031-407e-9237-be686b1c707b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:48.558250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:48.568700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 167/334] 10.222.0.45 () {58 vars in 1334 bytes} [Tue May 24 12:45:48 2022] PUT /identity/v3/system/users/8a74a957f39f475c8a19f8f364c46132/roles/4a71e72752894223a2b3fd53f728c211 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:50.357584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56290138-fb89-4cff-a7e8-ae4c033a8958 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:50.358087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56290138-fb89-4cff-a7e8-ae4c033a8958 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:50.358160 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56290138-fb89-4cff-a7e8-ae4c033a8958 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:50.359540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 168/335] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:50.367542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e7691-f862-435e-bf08-349447d24a52 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:50.367542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e7691-f862-435e-bf08-349447d24a52 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:50.367542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e7691-f862-435e-bf08-349447d24a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:50.384587 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-547e7691-f862-435e-bf08-349447d24a52 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:50.386077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-547e7691-f862-435e-bf08-349447d24a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:50.483029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 168/336] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:50.519123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56290138-fb89-4cff-a7e8-ae4c033a8958 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:50.638736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:50.648476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56290138-fb89-4cff-a7e8-ae4c033a8958 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:50.686304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678a8b7-f26b-45df-b242-5eceed80dd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:50.687118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678a8b7-f26b-45df-b242-5eceed80dd27 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:50.687404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678a8b7-f26b-45df-b242-5eceed80dd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:50.687683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678a8b7-f26b-45df-b242-5eceed80dd27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:50.700254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678a8b7-f26b-45df-b242-5eceed80dd27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-f678a8b7-f26b-45df-b242-5eceed80dd27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:50.708161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:50.709070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:50.709947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 169/337] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:50.716881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e7691-f862-435e-bf08-349447d24a52 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:50.856452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e7691-f862-435e-bf08-349447d24a52 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:50.892028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:50.897876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17919445-a985-4fa4-825c-fbff50197596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:50.898581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17919445-a985-4fa4-825c-fbff50197596 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:50.898874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17919445-a985-4fa4-825c-fbff50197596 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:50.899149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17919445-a985-4fa4-825c-fbff50197596 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:50.899908 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17919445-a985-4fa4-825c-fbff50197596 None admin] RBAC: Adding query filter params (name=member) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:50.900211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17919445-a985-4fa4-825c-fbff50197596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:50.901597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17919445-a985-4fa4-825c-fbff50197596 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:50.915627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 169/338] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:50.921155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678a8b7-f26b-45df-b242-5eceed80dd27 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:50.965303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d1f06-4802-4a2b-a990-d53027aefe3c None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:50.965587 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d1f06-4802-4a2b-a990-d53027aefe3c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:50.966088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d1f06-4802-4a2b-a990-d53027aefe3c None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:50.967952 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 170/339] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:50.973102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b70f-7aa2-4e70-b442-da6738e774ef None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:50.973634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b70f-7aa2-4e70-b442-da6738e774ef None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:50.974117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b70f-7aa2-4e70-b442-da6738e774ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:51.002007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-33b7b70f-7aa2-4e70-b442-da6738e774ef None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:51.003562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-33b7b70f-7aa2-4e70-b442-da6738e774ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:51.047051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678a8b7-f26b-45df-b242-5eceed80dd27 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:51.085142 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3aa904-ff9f-408f-b787-b9bf30ddc21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:51.086091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa904-ff9f-408f-b787-b9bf30ddc21f None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:51.086673 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa904-ff9f-408f-b787-b9bf30ddc21f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:51.087111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa904-ff9f-408f-b787-b9bf30ddc21f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:51.101724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:51.128575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 171/340] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:51.164419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3aa904-ff9f-408f-b787-b9bf30ddc21f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:51.166910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3aa904-ff9f-408f-b787-b9bf30ddc21f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:51.169868 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b70f-7aa2-4e70-b442-da6738e774ef None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:51.285012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 170/341] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:51.300200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3aa904-ff9f-408f-b787-b9bf30ddc21f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:51.329024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b70f-7aa2-4e70-b442-da6738e774ef None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:51.368050 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:51.374306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:51.391697 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5531f-ba5f-4d74-ae41-c3294de9f110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:51.392850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5531f-ba5f-4d74-ae41-c3294de9f110 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:51.392850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5531f-ba5f-4d74-ae41-c3294de9f110 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:51.393130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5531f-ba5f-4d74-ae41-c3294de9f110 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:51.408480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5531f-ba5f-4d74-ae41-c3294de9f110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-dcb5531f-ba5f-4d74-ae41-c3294de9f110 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:51.428988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:51.429968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:51.430849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 172/342] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:45:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:51.439811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5531f-ba5f-4d74-ae41-c3294de9f110 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:51.476445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3aa904-ff9f-408f-b787-b9bf30ddc21f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:51.551206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7e63d-403a-41e0-ac0c-d8e32982ebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:51.552127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7e63d-403a-41e0-ac0c-d8e32982ebea None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:51.552414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7e63d-403a-41e0-ac0c-d8e32982ebea None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:51.552693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7e63d-403a-41e0-ac0c-d8e32982ebea None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:51.558320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7e63d-403a-41e0-ac0c-d8e32982ebea None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:51.559957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7e63d-403a-41e0-ac0c-d8e32982ebea None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:51.562249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 171/343] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:51 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:51.588254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7e63d-403a-41e0-ac0c-d8e32982ebea None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:51.597644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5531f-ba5f-4d74-ae41-c3294de9f110 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:51.637852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258674c2-899f-4a83-b851-b602fdbf8ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:51.638576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258674c2-899f-4a83-b851-b602fdbf8ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:51.638861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258674c2-899f-4a83-b851-b602fdbf8ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:51.639137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258674c2-899f-4a83-b851-b602fdbf8ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:51.639896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258674c2-899f-4a83-b851-b602fdbf8ba2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:51.640206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258674c2-899f-4a83-b851-b602fdbf8ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:51.642118 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258674c2-899f-4a83-b851-b602fdbf8ba2 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:51.651091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 173/344] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:45:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:51.659564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258674c2-899f-4a83-b851-b602fdbf8ba2 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:51.733249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:51.799911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7e63d-403a-41e0-ac0c-d8e32982ebea None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:51.818796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:51.840453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258674c2-899f-4a83-b851-b602fdbf8ba2 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:51.859822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc60c58-4b85-4774-89cd-fed04976f43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:51.861333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc60c58-4b85-4774-89cd-fed04976f43d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:51.861621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc60c58-4b85-4774-89cd-fed04976f43d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:51.861896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc60c58-4b85-4774-89cd-fed04976f43d None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:51.877000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc60c58-4b85-4774-89cd-fed04976f43d None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:51.878578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc60c58-4b85-4774-89cd-fed04976f43d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:51.880399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 172/345] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:45:51 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:51.882959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec2452-251d-4bf5-a4e4-1f5e179ab52d None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:51.883249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec2452-251d-4bf5-a4e4-1f5e179ab52d None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:51.883575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec2452-251d-4bf5-a4e4-1f5e179ab52d None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:51.885307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 173/346] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:51.887677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec2452-251d-4bf5-a4e4-1f5e179ab52d None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:51.893927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f76495-7124-43ed-a9c1-6f24e32ec8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:51.894943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f76495-7124-43ed-a9c1-6f24e32ec8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:51.895382 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f76495-7124-43ed-a9c1-6f24e32ec8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:51.895806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f76495-7124-43ed-a9c1-6f24e32ec8b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:51.988583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f76495-7124-43ed-a9c1-6f24e32ec8b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:51.993182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f76495-7124-43ed-a9c1-6f24e32ec8b4 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:52.068028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec2452-251d-4bf5-a4e4-1f5e179ab52d None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:52.071971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:52.113802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa6bba-2acd-4345-b6c3-a52ca8ba1ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:52.114798 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa6bba-2acd-4345-b6c3-a52ca8ba1ebd None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:52.115079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa6bba-2acd-4345-b6c3-a52ca8ba1ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:52.115354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa6bba-2acd-4345-b6c3-a52ca8ba1ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:52.116209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa6bba-2acd-4345-b6c3-a52ca8ba1ebd None admin] RBAC: Adding query filter params (group.id=d497ceffe4bf48c3aa012faf1295a02f, role.id=27d8d0ea0b83488a95a9141d439247f6, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:52.116522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa6bba-2acd-4345-b6c3-a52ca8ba1ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:52.117992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa6bba-2acd-4345-b6c3-a52ca8ba1ebd None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:52.125034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 174/347] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 12:45:51 2022] GET /identity/v3/role_assignments?group.id=d497ceffe4bf48c3aa012faf1295a02f&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:52.126938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c10a7-1e5d-4a89-a786-9219c03e9d9e None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:52.127229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c10a7-1e5d-4a89-a786-9219c03e9d9e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:52.127506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c10a7-1e5d-4a89-a786-9219c03e9d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:52.145043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-d34c10a7-1e5d-4a89-a786-9219c03e9d9e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:52.146220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d34c10a7-1e5d-4a89-a786-9219c03e9d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:52.149299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 174/348] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:45:52.150983 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98bc8a-7466-4923-a3d1-56cd822bb145 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:52.151497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98bc8a-7466-4923-a3d1-56cd822bb145 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:52.151974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98bc8a-7466-4923-a3d1-56cd822bb145 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:52.153910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 175/349] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:52.156021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98bc8a-7466-4923-a3d1-56cd822bb145 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:52.192765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:52.300742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 175/350] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:52.302531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54095b5c-021f-4a49-9b02-84f57791759b None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:52.305092 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54095b5c-021f-4a49-9b02-84f57791759b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:52.305832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98bc8a-7466-4923-a3d1-56cd822bb145 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:52.307805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54095b5c-021f-4a49-9b02-84f57791759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:52.343587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-54095b5c-021f-4a49-9b02-84f57791759b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:52.344858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-54095b5c-021f-4a49-9b02-84f57791759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:52.366636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526fb26-db65-4071-a76f-f2e1e971b4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:52.367536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526fb26-db65-4071-a76f-f2e1e971b4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:52.367827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526fb26-db65-4071-a76f-f2e1e971b4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:52.368117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526fb26-db65-4071-a76f-f2e1e971b4e2 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:52.370149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526fb26-db65-4071-a76f-f2e1e971b4e2 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:52.371671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526fb26-db65-4071-a76f-f2e1e971b4e2 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:52.375923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 176/351] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:52 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:52.376677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526fb26-db65-4071-a76f-f2e1e971b4e2 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:52.406391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:52.481212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 176/352] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:52.482083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54095b5c-021f-4a49-9b02-84f57791759b None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:52.569347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526fb26-db65-4071-a76f-f2e1e971b4e2 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:52.615024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:52.616854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d386a-65a6-4820-a088-ae2ce0d7dee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:52.617834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d386a-65a6-4820-a088-ae2ce0d7dee4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:52.618125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d386a-65a6-4820-a088-ae2ce0d7dee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:52.618992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d386a-65a6-4820-a088-ae2ce0d7dee4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:52.624335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54095b5c-021f-4a49-9b02-84f57791759b None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:52.632453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d386a-65a6-4820-a088-ae2ce0d7dee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-f31d386a-65a6-4820-a088-ae2ce0d7dee4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:52.643345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:52.644290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:52.645133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 177/353] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:52.645818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f76ec1-b67a-4c4c-a4ba-7984f367652c None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:52.646694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f76ec1-b67a-4c4c-a4ba-7984f367652c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:52.646694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f76ec1-b67a-4c4c-a4ba-7984f367652c None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:52.648814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 178/354] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:52.648814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f76ec1-b67a-4c4c-a4ba-7984f367652c None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:52.681061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd8240-af9f-4f9c-9837-fb368eab612a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:52.682344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd8240-af9f-4f9c-9837-fb368eab612a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:52.682863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd8240-af9f-4f9c-9837-fb368eab612a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:52.683343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd8240-af9f-4f9c-9837-fb368eab612a None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:52.684729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd8240-af9f-4f9c-9837-fb368eab612a None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:52.686423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd8240-af9f-4f9c-9837-fb368eab612a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:52.690682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 177/355] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:45:52 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:52.692300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d7624-1793-4b45-bda5-aad356bb29d1 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:52.692811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d7624-1793-4b45-bda5-aad356bb29d1 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:52.693287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d7624-1793-4b45-bda5-aad356bb29d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:52.711457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-673d7624-1793-4b45-bda5-aad356bb29d1 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:52.713217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-673d7624-1793-4b45-bda5-aad356bb29d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:52.764294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:52.807289 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 178/356] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:52.807970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d7624-1793-4b45-bda5-aad356bb29d1 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:52.808787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f76ec1-b67a-4c4c-a4ba-7984f367652c None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:52.852291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b0a41-edb7-4489-bbd9-6b938935ca0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:52.853110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b0a41-edb7-4489-bbd9-6b938935ca0b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:52.853464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b0a41-edb7-4489-bbd9-6b938935ca0b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:52.853812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b0a41-edb7-4489-bbd9-6b938935ca0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:52.868032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b0a41-edb7-4489-bbd9-6b938935ca0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-6e0b0a41-edb7-4489-bbd9-6b938935ca0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:52.876753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:52.877686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:52.879109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 179/357] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:52.880631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b0a41-edb7-4489-bbd9-6b938935ca0b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:52.932238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:52.943676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d7624-1793-4b45-bda5-aad356bb29d1 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:52.989287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec1a21-2010-44c8-8cfa-9b40b5b71d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:52.990002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec1a21-2010-44c8-8cfa-9b40b5b71d64 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:52.990315 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec1a21-2010-44c8-8cfa-9b40b5b71d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:52.990604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec1a21-2010-44c8-8cfa-9b40b5b71d64 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:52.991353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec1a21-2010-44c8-8cfa-9b40b5b71d64 None admin] RBAC: Adding query filter params (name=member) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:52.991665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec1a21-2010-44c8-8cfa-9b40b5b71d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:52.993507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec1a21-2010-44c8-8cfa-9b40b5b71d64 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:53.003723 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 179/358] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:53.004736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec1a21-2010-44c8-8cfa-9b40b5b71d64 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:53.015918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b0a41-edb7-4489-bbd9-6b938935ca0b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:53.058405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:53.068839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0dbac-730e-4901-b92c-141be8482641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:53.070716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0dbac-730e-4901-b92c-141be8482641 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:53.070716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0dbac-730e-4901-b92c-141be8482641 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:53.070716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0dbac-730e-4901-b92c-141be8482641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:53.071041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0dbac-730e-4901-b92c-141be8482641 None admin] RBAC: Adding query filter params (group.id=d497ceffe4bf48c3aa012faf1295a02f, role.id=c7b7d3b4dd944936851dee41ead5dc99, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:53.071363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0dbac-730e-4901-b92c-141be8482641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:53.072832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0dbac-730e-4901-b92c-141be8482641 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:53.079089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 180/359] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 12:45:52 2022] GET /identity/v3/role_assignments?group.id=d497ceffe4bf48c3aa012faf1295a02f&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=c7b7d3b4dd944936851dee41ead5dc99 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:53.080399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0dbac-730e-4901-b92c-141be8482641 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:53.147845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec1a21-2010-44c8-8cfa-9b40b5b71d64 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:53.198226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142d14b-a5f2-46f1-a77a-a55ff7fcb6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:53.199402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142d14b-a5f2-46f1-a77a-a55ff7fcb6ec None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:53.199687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142d14b-a5f2-46f1-a77a-a55ff7fcb6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:53.199961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142d14b-a5f2-46f1-a77a-a55ff7fcb6ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:53.209565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0dbac-730e-4901-b92c-141be8482641 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:53.213633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142d14b-a5f2-46f1-a77a-a55ff7fcb6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-b142d14b-a5f2-46f1-a77a-a55ff7fcb6ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:53.223711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:53.224694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:53.225616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 180/360] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:45:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:53.226389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ae336-1b9b-40d8-b4ec-30012056ae18 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:53.226695 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ae336-1b9b-40d8-b4ec-30012056ae18 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:53.227260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ae336-1b9b-40d8-b4ec-30012056ae18 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:53.228749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 181/361] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:53.229249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ae336-1b9b-40d8-b4ec-30012056ae18 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:53.257222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b915ae6-e024-40f3-84cb-3dd5d6b08751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:53.257917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b915ae6-e024-40f3-84cb-3dd5d6b08751 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:53.258199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b915ae6-e024-40f3-84cb-3dd5d6b08751 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:53.258510 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b915ae6-e024-40f3-84cb-3dd5d6b08751 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:53.259276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b915ae6-e024-40f3-84cb-3dd5d6b08751 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:53.259585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b915ae6-e024-40f3-84cb-3dd5d6b08751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:53.259807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:53.260994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b915ae6-e024-40f3-84cb-3dd5d6b08751 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:53.274748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 181/362] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:53.275281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6748f85-5d3f-4826-868b-1967525be1e4 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:53.275574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6748f85-5d3f-4826-868b-1967525be1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:53.275919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6748f85-5d3f-4826-868b-1967525be1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:53.294313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-c6748f85-5d3f-4826-868b-1967525be1e4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:53.295619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c6748f85-5d3f-4826-868b-1967525be1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:53.370762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:53.394162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 182/363] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:53.395053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6748f85-5d3f-4826-868b-1967525be1e4 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:53.410687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ae336-1b9b-40d8-b4ec-30012056ae18 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:53.457410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34631860-de86-42e7-9ab1-1df5cf00178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:53.458382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34631860-de86-42e7-9ab1-1df5cf00178e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:53.458677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34631860-de86-42e7-9ab1-1df5cf00178e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:53.458961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34631860-de86-42e7-9ab1-1df5cf00178e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:53.508381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6748f85-5d3f-4826-868b-1967525be1e4 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:53.547579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e8cf9-4133-41f4-b4bb-04076f28637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:53.548343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e8cf9-4133-41f4-b4bb-04076f28637e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:53.548633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e8cf9-4133-41f4-b4bb-04076f28637e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:53.548903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e8cf9-4133-41f4-b4bb-04076f28637e None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:53.549656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e8cf9-4133-41f4-b4bb-04076f28637e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:53.550018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e8cf9-4133-41f4-b4bb-04076f28637e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:53.552030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34631860-de86-42e7-9ab1-1df5cf00178e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:53.552479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e8cf9-4133-41f4-b4bb-04076f28637e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:53.553766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34631860-de86-42e7-9ab1-1df5cf00178e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:53.562210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 183/364] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:45:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:53.563077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e8cf9-4133-41f4-b4bb-04076f28637e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:53.576248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:53.631833 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:53.692821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 182/365] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:53.693599 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34631860-de86-42e7-9ab1-1df5cf00178e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:53.702601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e8cf9-4133-41f4-b4bb-04076f28637e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:53.749760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be18d9a-571e-4f46-a223-f86c76f16cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:53.751586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be18d9a-571e-4f46-a223-f86c76f16cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:53.751873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be18d9a-571e-4f46-a223-f86c76f16cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:53.752234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be18d9a-571e-4f46-a223-f86c76f16cf4 None admin] PATH_INFO: `/v3/users/8a74a957f39f475c8a19f8f364c46132` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:53.754043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be18d9a-571e-4f46-a223-f86c76f16cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=8a74a957f39f475c8a19f8f364c46132)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:53.755767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be18d9a-571e-4f46-a223-f86c76f16cf4 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:53.758101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 184/366] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:45:53 2022] GET /identity/v3/users/8a74a957f39f475c8a19f8f364c46132 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:53.758923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be18d9a-571e-4f46-a223-f86c76f16cf4 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:53.813567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34631860-de86-42e7-9ab1-1df5cf00178e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:53.833923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:53.855375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a7ed9-57ab-4c0b-86bd-b20b946a9a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:53.856172 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a7ed9-57ab-4c0b-86bd-b20b946a9a64 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:53.856454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a7ed9-57ab-4c0b-86bd-b20b946a9a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:53.856725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a7ed9-57ab-4c0b-86bd-b20b946a9a64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:53.867917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a7ed9-57ab-4c0b-86bd-b20b946a9a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-b07a7ed9-57ab-4c0b-86bd-b20b946a9a64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:53.875848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:53.878450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 183/367] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:45:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:53.880450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a7ed9-57ab-4c0b-86bd-b20b946a9a64 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:53.890820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be18d9a-571e-4f46-a223-f86c76f16cf4 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:53.915693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:53.935594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158402ca-6d97-4ca3-99cb-a7baab0eb941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:53.936266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158402ca-6d97-4ca3-99cb-a7baab0eb941 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:53.936552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158402ca-6d97-4ca3-99cb-a7baab0eb941 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:53.936831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158402ca-6d97-4ca3-99cb-a7baab0eb941 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:53.995113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a7ed9-57ab-4c0b-86bd-b20b946a9a64 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:54.038987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158402ca-6d97-4ca3-99cb-a7baab0eb941 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:54.040865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158402ca-6d97-4ca3-99cb-a7baab0eb941 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:54.042141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e395462-4732-4707-9aa4-f6f827953dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:54.043136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e395462-4732-4707-9aa4-f6f827953dcf None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:54.043521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e395462-4732-4707-9aa4-f6f827953dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:54.043908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e395462-4732-4707-9aa4-f6f827953dcf None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:54.046986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e395462-4732-4707-9aa4-f6f827953dcf None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:54.049083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e395462-4732-4707-9aa4-f6f827953dcf None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:54.051296 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 184/368] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:53 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:54.052147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e395462-4732-4707-9aa4-f6f827953dcf None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:54.114550 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:54.173982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 185/369] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:54.174958 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158402ca-6d97-4ca3-99cb-a7baab0eb941 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:54.187944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e395462-4732-4707-9aa4-f6f827953dcf None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:54.194024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:54.237979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54299153-a014-4653-9e4c-1dd073cf8758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:54.239300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54299153-a014-4653-9e4c-1dd073cf8758 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:54.239597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54299153-a014-4653-9e4c-1dd073cf8758 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:54.239884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54299153-a014-4653-9e4c-1dd073cf8758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:54.240899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54299153-a014-4653-9e4c-1dd073cf8758 None admin] RBAC: Adding query filter params (role.id=4a71e72752894223a2b3fd53f728c211, user.id=8a74a957f39f475c8a19f8f364c46132, scope.system=all) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:54.241217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54299153-a014-4653-9e4c-1dd073cf8758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:54.243328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54299153-a014-4653-9e4c-1dd073cf8758 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:54.259949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 185/370] 10.222.0.45 () {56 vars in 1308 bytes} [Tue May 24 12:45:54 2022] GET /identity/v3/role_assignments?user.id=8a74a957f39f475c8a19f8f364c46132&scope.system=all&role.id=4a71e72752894223a2b3fd53f728c211 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:54.261099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54299153-a014-4653-9e4c-1dd073cf8758 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:54.287426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158402ca-6d97-4ca3-99cb-a7baab0eb941 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:54.336020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d8a2c-07b8-4fe3-a292-887e8de25e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:54.336727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8a2c-07b8-4fe3-a292-887e8de25e17 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:54.337167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8a2c-07b8-4fe3-a292-887e8de25e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:54.337457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8a2c-07b8-4fe3-a292-887e8de25e17 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:54.338237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d8a2c-07b8-4fe3-a292-887e8de25e17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:54.338695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d8a2c-07b8-4fe3-a292-887e8de25e17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:54.340110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d8a2c-07b8-4fe3-a292-887e8de25e17 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:54.349645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 186/371] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:54.350556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d8a2c-07b8-4fe3-a292-887e8de25e17 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:54.404641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54299153-a014-4653-9e4c-1dd073cf8758 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:54.406841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:54.461366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe308d-d6a0-4d41-98cf-c98bc9d11be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:54.462574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:54.463020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe308d-d6a0-4d41-98cf-c98bc9d11be4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:54.463020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe308d-d6a0-4d41-98cf-c98bc9d11be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:54.463216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe308d-d6a0-4d41-98cf-c98bc9d11be4 None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:54.465322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe308d-d6a0-4d41-98cf-c98bc9d11be4 None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:54.466631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe308d-d6a0-4d41-98cf-c98bc9d11be4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:54.468440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 186/372] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:45:54 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:54.469226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe308d-d6a0-4d41-98cf-c98bc9d11be4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:54.488414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d8a2c-07b8-4fe3-a292-887e8de25e17 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:54.527128 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7528b-14db-4728-bcbc-30c75c6ee762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:54.528253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7528b-14db-4728-bcbc-30c75c6ee762 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:54.528701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7528b-14db-4728-bcbc-30c75c6ee762 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:54.529138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7528b-14db-4728-bcbc-30c75c6ee762 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:54.530919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7528b-14db-4728-bcbc-30c75c6ee762 None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:54.533142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7528b-14db-4728-bcbc-30c75c6ee762 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:54.535641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 187/373] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:54 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:54.536574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7528b-14db-4728-bcbc-30c75c6ee762 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:54.591086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe308d-d6a0-4d41-98cf-c98bc9d11be4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:54.631233 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df4c64-3233-41b9-8a57-2f086530da9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:54.632017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df4c64-3233-41b9-8a57-2f086530da9d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:54.632311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df4c64-3233-41b9-8a57-2f086530da9d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:54.632588 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df4c64-3233-41b9-8a57-2f086530da9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:54.667486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7528b-14db-4728-bcbc-30c75c6ee762 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:54.677527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:54.714247 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df4c64-3233-41b9-8a57-2f086530da9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:54.715648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c8af0-6c90-4616-b206-0aeabddbaae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:54.715941 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df4c64-3233-41b9-8a57-2f086530da9d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:54.716803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c8af0-6c90-4616-b206-0aeabddbaae3 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:54.717232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c8af0-6c90-4616-b206-0aeabddbaae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:54.717653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c8af0-6c90-4616-b206-0aeabddbaae3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:54.718853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c8af0-6c90-4616-b206-0aeabddbaae3 None admin] RBAC: Adding query filter params (group.id=d497ceffe4bf48c3aa012faf1295a02f, role.id=27d8d0ea0b83488a95a9141d439247f6, scope.project.id=197b0ba2357a405ca5849973bb209b46) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:54.719313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c8af0-6c90-4616-b206-0aeabddbaae3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:54.721783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c8af0-6c90-4616-b206-0aeabddbaae3 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:54.729426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 188/374] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 12:45:54 2022] GET /identity/v3/role_assignments?group.id=d497ceffe4bf48c3aa012faf1295a02f&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:54.730785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c8af0-6c90-4616-b206-0aeabddbaae3 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:54.753983 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:54.868290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 187/375] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:45:54.875367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df4c64-3233-41b9-8a57-2f086530da9d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:54.878806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c8af0-6c90-4616-b206-0aeabddbaae3 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:54.950259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19382a0-913b-49be-805b-6a47edaf61ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:54.951465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19382a0-913b-49be-805b-6a47edaf61ad None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:54.951920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19382a0-913b-49be-805b-6a47edaf61ad None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:54.952340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19382a0-913b-49be-805b-6a47edaf61ad None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:54.954097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19382a0-913b-49be-805b-6a47edaf61ad None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:54.955807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19382a0-913b-49be-805b-6a47edaf61ad None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:54.957678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 189/376] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:45:54 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:54.963856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19382a0-913b-49be-805b-6a47edaf61ad None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:55.018657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:55.085265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df4c64-3233-41b9-8a57-2f086530da9d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:55.089249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:55.116775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19382a0-913b-49be-805b-6a47edaf61ad None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:55.153041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5b5d8-1b35-4bee-92a1-ced8ab0397a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:55.154133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5b5d8-1b35-4bee-92a1-ced8ab0397a3 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:55.154599 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5b5d8-1b35-4bee-92a1-ced8ab0397a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:55.155015 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5b5d8-1b35-4bee-92a1-ced8ab0397a3 None admin] PATH_INFO: `/v3/projects/26aa7dae9c0c4f5a892a6fcef95c15ea` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:55.156431 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5b5d8-1b35-4bee-92a1-ced8ab0397a3 None admin] RBAC: Authorizing `identity:get_project(project_id=26aa7dae9c0c4f5a892a6fcef95c15ea)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:55.160250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5b5d8-1b35-4bee-92a1-ced8ab0397a3 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:55.162523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 188/377] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:54 2022] GET /identity/v3/projects/26aa7dae9c0c4f5a892a6fcef95c15ea => generated 344 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:55.169814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5b5d8-1b35-4bee-92a1-ced8ab0397a3 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:55.186641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a7254-b2fc-49e4-bd6e-a18787c8d628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:55.187312 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a7254-b2fc-49e4-bd6e-a18787c8d628 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:55.187593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a7254-b2fc-49e4-bd6e-a18787c8d628 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:55.187867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a7254-b2fc-49e4-bd6e-a18787c8d628 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:55.189236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a7254-b2fc-49e4-bd6e-a18787c8d628 None admin] RBAC: Adding query filter params (group.id=d497ceffe4bf48c3aa012faf1295a02f, role.id=c7b7d3b4dd944936851dee41ead5dc99, scope.project.id=197b0ba2357a405ca5849973bb209b46) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:55.189550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a7254-b2fc-49e4-bd6e-a18787c8d628 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:55.191024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a7254-b2fc-49e4-bd6e-a18787c8d628 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:55.200425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 190/378] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 12:45:54 2022] GET /identity/v3/role_assignments?group.id=d497ceffe4bf48c3aa012faf1295a02f&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=c7b7d3b4dd944936851dee41ead5dc99 => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:55.289861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be5bbb-8c4d-47e7-9538-03f73a0ee802 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:55.290215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be5bbb-8c4d-47e7-9538-03f73a0ee802 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:55.290494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be5bbb-8c4d-47e7-9538-03f73a0ee802 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:55.292480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 191/379] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:45:55.299655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213db9a9-d1f2-44ab-82c5-ead24b252209 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:55.299965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213db9a9-d1f2-44ab-82c5-ead24b252209 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:55.300198 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213db9a9-d1f2-44ab-82c5-ead24b252209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:55.327398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-213db9a9-d1f2-44ab-82c5-ead24b252209 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:55.328600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-213db9a9-d1f2-44ab-82c5-ead24b252209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:55.358477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5b5d8-1b35-4bee-92a1-ced8ab0397a3 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:55.446236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbbd1d-65b7-4225-96dd-9fe53c47a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:55.447199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbbd1d-65b7-4225-96dd-9fe53c47a064 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:55.447480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbbd1d-65b7-4225-96dd-9fe53c47a064 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:55.447749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbbd1d-65b7-4225-96dd-9fe53c47a064 None admin] PATH_INFO: `/v3/groups/5ddaebc8378a49fdb891f468fc7d8c4f` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:55.460729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbbd1d-65b7-4225-96dd-9fe53c47a064 None admin] RBAC: Authorizing `identity:get_group(group_id=5ddaebc8378a49fdb891f468fc7d8c4f)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:55.462308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbbd1d-65b7-4225-96dd-9fe53c47a064 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:55.464072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 189/380] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:45:55 2022] GET /identity/v3/groups/5ddaebc8378a49fdb891f468fc7d8c4f => generated 229 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:55.483704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbbd1d-65b7-4225-96dd-9fe53c47a064 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:55.486568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:55.495671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 192/381] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:55.530840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213db9a9-d1f2-44ab-82c5-ead24b252209 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:55.602376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbbd1d-65b7-4225-96dd-9fe53c47a064 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:55.639382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447cb63-95c1-4d1e-8d4f-6eb7b5c5adcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:55.640436 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447cb63-95c1-4d1e-8d4f-6eb7b5c5adcf None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:55.640935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447cb63-95c1-4d1e-8d4f-6eb7b5c5adcf None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:55.642017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447cb63-95c1-4d1e-8d4f-6eb7b5c5adcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:55.643172 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447cb63-95c1-4d1e-8d4f-6eb7b5c5adcf None admin] RBAC: Adding query filter params (group.id=5ddaebc8378a49fdb891f468fc7d8c4f, role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=26aa7dae9c0c4f5a892a6fcef95c15ea) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:55.643716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447cb63-95c1-4d1e-8d4f-6eb7b5c5adcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:55.645417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447cb63-95c1-4d1e-8d4f-6eb7b5c5adcf None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:55.651529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:55.653702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 190/382] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 12:45:55 2022] GET /identity/v3/role_assignments?group.id=5ddaebc8378a49fdb891f468fc7d8c4f&scope.project.id=26aa7dae9c0c4f5a892a6fcef95c15ea&role.id=0c473678b281495f86f58b077be04e1b => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:55.692888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213db9a9-d1f2-44ab-82c5-ead24b252209 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:55.740024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279bbba-6051-460b-82ab-a822ac25ddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:55.740862 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279bbba-6051-460b-82ab-a822ac25ddf0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:55.741145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279bbba-6051-460b-82ab-a822ac25ddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:55.741418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279bbba-6051-460b-82ab-a822ac25ddf0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:55.753951 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279bbba-6051-460b-82ab-a822ac25ddf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-f279bbba-6051-460b-82ab-a822ac25ddf0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:55.762367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:55.763409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:55.764220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 193/383] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:55.770953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447cb63-95c1-4d1e-8d4f-6eb7b5c5adcf None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:55.856658 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:55.858160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059e98e-6231-4a32-82ff-32d41d4a77fe None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:55.858492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059e98e-6231-4a32-82ff-32d41d4a77fe None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:55.858774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059e98e-6231-4a32-82ff-32d41d4a77fe None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:55.861206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 194/384] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:55.867104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ee782-3b63-40f6-bd81-a60e3e29c021 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:55.867439 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ee782-3b63-40f6-bd81-a60e3e29c021 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:55.867670 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ee782-3b63-40f6-bd81-a60e3e29c021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:55.901570 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447cb63-95c1-4d1e-8d4f-6eb7b5c5adcf None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:55.912159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1a3ee782-3b63-40f6-bd81-a60e3e29c021 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:55.912653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1a3ee782-3b63-40f6-bd81-a60e3e29c021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:55.949801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d59f3-8e90-447b-a63b-f3d762878069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:55.950851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d59f3-8e90-447b-a63b-f3d762878069 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:55.951332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d59f3-8e90-447b-a63b-f3d762878069 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:55.952112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d59f3-8e90-447b-a63b-f3d762878069 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:55.953070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d59f3-8e90-447b-a63b-f3d762878069 None admin] RBAC: Adding query filter params (name=member) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:55.953593 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d59f3-8e90-447b-a63b-f3d762878069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:55.956378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d59f3-8e90-447b-a63b-f3d762878069 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:55.971498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 191/385] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:55.977618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d59f3-8e90-447b-a63b-f3d762878069 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:56.062441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 195/386] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:56.080930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ee782-3b63-40f6-bd81-a60e3e29c021 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:56.169601 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d59f3-8e90-447b-a63b-f3d762878069 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:56.208768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:56.225678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:56.238645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8abe2-09f2-42c4-ae0d-5789ed27450e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:56.239571 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8abe2-09f2-42c4-ae0d-5789ed27450e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:56.239856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8abe2-09f2-42c4-ae0d-5789ed27450e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:56.240138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8abe2-09f2-42c4-ae0d-5789ed27450e None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:56.241454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8abe2-09f2-42c4-ae0d-5789ed27450e None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:56.242977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8abe2-09f2-42c4-ae0d-5789ed27450e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:56.249756 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 192/387] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:45:55 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:56.255903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8abe2-09f2-42c4-ae0d-5789ed27450e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:56.274532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ee782-3b63-40f6-bd81-a60e3e29c021 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:56.327665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71058834-b1f0-498f-ad18-b78fcd71a1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:56.328545 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71058834-b1f0-498f-ad18-b78fcd71a1ad None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:56.329442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71058834-b1f0-498f-ad18-b78fcd71a1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:56.329696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71058834-b1f0-498f-ad18-b78fcd71a1ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:56.350198 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71058834-b1f0-498f-ad18-b78fcd71a1ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-71058834-b1f0-498f-ad18-b78fcd71a1ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:56.358938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:56.360658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:56.362243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:45:56.370312 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 196/388] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:45:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:56.403397 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71058834-b1f0-498f-ad18-b78fcd71a1ad None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:56.478376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8abe2-09f2-42c4-ae0d-5789ed27450e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:56.525855 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24031ee-579f-4c79-b4cc-3d8a7eb79db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:56.526873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24031ee-579f-4c79-b4cc-3d8a7eb79db4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:56.527171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24031ee-579f-4c79-b4cc-3d8a7eb79db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:56.527455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24031ee-579f-4c79-b4cc-3d8a7eb79db4 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:56.534310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24031ee-579f-4c79-b4cc-3d8a7eb79db4 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:56.534310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24031ee-579f-4c79-b4cc-3d8a7eb79db4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:56.534827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 193/389] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:56 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:56.540978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24031ee-579f-4c79-b4cc-3d8a7eb79db4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:56.614702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:56.626557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71058834-b1f0-498f-ad18-b78fcd71a1ad None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:56.659071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24031ee-579f-4c79-b4cc-3d8a7eb79db4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:56.661209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:56.677517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8930a9-9945-4cbf-acbc-28e24a87244c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:56.678376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8930a9-9945-4cbf-acbc-28e24a87244c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:56.678770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8930a9-9945-4cbf-acbc-28e24a87244c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:56.679151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8930a9-9945-4cbf-acbc-28e24a87244c None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:56.680132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8930a9-9945-4cbf-acbc-28e24a87244c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:56.680545 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8930a9-9945-4cbf-acbc-28e24a87244c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:56.682125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8930a9-9945-4cbf-acbc-28e24a87244c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:56.692683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 197/390] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:45:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:56.699786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8930a9-9945-4cbf-acbc-28e24a87244c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:56.702833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf7cce-5a0e-46f6-b02c-63e600192bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:56.704123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf7cce-5a0e-46f6-b02c-63e600192bca None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:56.704666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf7cce-5a0e-46f6-b02c-63e600192bca None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:56.705171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf7cce-5a0e-46f6-b02c-63e600192bca None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/groups/d497ceffe4bf48c3aa012faf1295a02f/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:56.712865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf7cce-5a0e-46f6-b02c-63e600192bca None admin] RBAC: Authorizing `identity:create_grant(project_id=a75c6b7e46c24ae8adacf3d5669e7162, group_id=d497ceffe4bf48c3aa012faf1295a02f, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:56.715506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf7cce-5a0e-46f6-b02c-63e600192bca None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:56.745637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 194/391] 10.222.0.45 () {58 vars in 1442 bytes} [Tue May 24 12:45:56 2022] PUT /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/groups/d497ceffe4bf48c3aa012faf1295a02f/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:56.903400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8930a9-9945-4cbf-acbc-28e24a87244c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:56.963622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81efe6-3864-42aa-97f4-2d06b2b11f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:56.964512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81efe6-3864-42aa-97f4-2d06b2b11f1d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:56.964793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81efe6-3864-42aa-97f4-2d06b2b11f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:56.965066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81efe6-3864-42aa-97f4-2d06b2b11f1d None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:56.966263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a81efe6-3864-42aa-97f4-2d06b2b11f1d None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:56.967729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a81efe6-3864-42aa-97f4-2d06b2b11f1d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:56.969721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 198/392] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:45:56 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:56.976895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf7cce-5a0e-46f6-b02c-63e600192bca None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:57.110609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf7cce-5a0e-46f6-b02c-63e600192bca None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:57.137237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:57.157566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9e268-641e-447d-8fa3-4582c4cd9172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:57.158505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9e268-641e-447d-8fa3-4582c4cd9172 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:57.158790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9e268-641e-447d-8fa3-4582c4cd9172 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:57.159065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9e268-641e-447d-8fa3-4582c4cd9172 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:57.163096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9e268-641e-447d-8fa3-4582c4cd9172 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:57.164631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9e268-641e-447d-8fa3-4582c4cd9172 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:57.166690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 195/393] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:56 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:57.172491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81efe6-3864-42aa-97f4-2d06b2b11f1d None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:57.243250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:57.315535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81efe6-3864-42aa-97f4-2d06b2b11f1d None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:57.370982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabefb8-404a-428c-ab56-977baa1e77e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:57.372216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabefb8-404a-428c-ab56-977baa1e77e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:57.372701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabefb8-404a-428c-ab56-977baa1e77e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:57.373174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabefb8-404a-428c-ab56-977baa1e77e5 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/groups/d497ceffe4bf48c3aa012faf1295a02f/roles/c7b7d3b4dd944936851dee41ead5dc99` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:57.379586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabefb8-404a-428c-ab56-977baa1e77e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a75c6b7e46c24ae8adacf3d5669e7162, group_id=d497ceffe4bf48c3aa012faf1295a02f, role_id=c7b7d3b4dd944936851dee41ead5dc99)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:57.381268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabefb8-404a-428c-ab56-977baa1e77e5 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:57.410184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 199/394] 10.222.0.45 () {58 vars in 1442 bytes} [Tue May 24 12:45:57 2022] PUT /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162/groups/d497ceffe4bf48c3aa012faf1295a02f/roles/c7b7d3b4dd944936851dee41ead5dc99 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:45:57.856630 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea9508-ead0-476d-a796-ce077e00d2fe None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:57.857348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea9508-ead0-476d-a796-ce077e00d2fe None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:57.857348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea9508-ead0-476d-a796-ce077e00d2fe None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:57.858962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 196/395] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:57.865345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832de543-dff2-4eb5-b3bd-1802c27a61f6 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:57.865924 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832de543-dff2-4eb5-b3bd-1802c27a61f6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:57.865924 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832de543-dff2-4eb5-b3bd-1802c27a61f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:57.885373 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-832de543-dff2-4eb5-b3bd-1802c27a61f6 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:57.887001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-832de543-dff2-4eb5-b3bd-1802c27a61f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:57.971082 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 200/396] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:58.019669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea9508-ead0-476d-a796-ce077e00d2fe None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:58.159674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea9508-ead0-476d-a796-ce077e00d2fe None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:58.204353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d8637-ddfe-4cc3-8f0f-9359a9d8d02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:58.205155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d8637-ddfe-4cc3-8f0f-9359a9d8d02d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:58.205441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d8637-ddfe-4cc3-8f0f-9359a9d8d02d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:58.205721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d8637-ddfe-4cc3-8f0f-9359a9d8d02d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:58.219014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d8637-ddfe-4cc3-8f0f-9359a9d8d02d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-801d8637-ddfe-4cc3-8f0f-9359a9d8d02d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:58.231513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:58.234369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:58.236983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 197/397] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:58.243980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832de543-dff2-4eb5-b3bd-1802c27a61f6 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:58.333229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:58.382380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eda36e-b922-403d-b8fb-fcba118ff0f6 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:58.382703 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eda36e-b922-403d-b8fb-fcba118ff0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:58.382951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eda36e-b922-403d-b8fb-fcba118ff0f6 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:58.384605 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 198/398] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:58.390702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c38348-ab24-422f-8b16-942b7a415a18 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:58.390995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c38348-ab24-422f-8b16-942b7a415a18 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:58.391822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c38348-ab24-422f-8b16-942b7a415a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:58.419571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832de543-dff2-4eb5-b3bd-1802c27a61f6 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:58.421674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b2c38348-ab24-422f-8b16-942b7a415a18 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:58.423548 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b2c38348-ab24-422f-8b16-942b7a415a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:58.464089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c8a98-b960-4eed-8340-7512f5674169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:58.464768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c8a98-b960-4eed-8340-7512f5674169 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:58.465051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c8a98-b960-4eed-8340-7512f5674169 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:58.465543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c8a98-b960-4eed-8340-7512f5674169 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:58.466326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c8a98-b960-4eed-8340-7512f5674169 None admin] RBAC: Adding query filter params (name=member) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:58.466644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c8a98-b960-4eed-8340-7512f5674169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:58.468020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c8a98-b960-4eed-8340-7512f5674169 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:58.477420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 201/399] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:45:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:58.483164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c8a98-b960-4eed-8340-7512f5674169 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:58.522789 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:58.559597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 199/400] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:45:58.596453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c38348-ab24-422f-8b16-942b7a415a18 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:58.597015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c8a98-b960-4eed-8340-7512f5674169 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:58.604616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:58.634028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b18d2-695f-4db6-934e-83e3728759b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:58.634932 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b18d2-695f-4db6-934e-83e3728759b1 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:58.635216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b18d2-695f-4db6-934e-83e3728759b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:58.635495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b18d2-695f-4db6-934e-83e3728759b1 None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:58.637004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b18d2-695f-4db6-934e-83e3728759b1 None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:58.638527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b18d2-695f-4db6-934e-83e3728759b1 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:58.640635 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 202/401] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:45:58 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:58.646714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b18d2-695f-4db6-934e-83e3728759b1 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:58.755175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b18d2-695f-4db6-934e-83e3728759b1 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:58.795325 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c38348-ab24-422f-8b16-942b7a415a18 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:58.795651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dc6b5-bb8f-4e76-a7ad-1f636232ed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:58.796370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6b5-bb8f-4e76-a7ad-1f636232ed3a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:58.796657 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6b5-bb8f-4e76-a7ad-1f636232ed3a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:58.797053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6b5-bb8f-4e76-a7ad-1f636232ed3a None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:58.798480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dc6b5-bb8f-4e76-a7ad-1f636232ed3a None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:58.799949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dc6b5-bb8f-4e76-a7ad-1f636232ed3a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:58.801960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 203/402] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:58 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:58.807484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dc6b5-bb8f-4e76-a7ad-1f636232ed3a None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:58.845655 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5550ee9-81cf-4c41-95d4-179e791fddfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:58.849013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5550ee9-81cf-4c41-95d4-179e791fddfd None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:58.849732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5550ee9-81cf-4c41-95d4-179e791fddfd None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:58.850412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5550ee9-81cf-4c41-95d4-179e791fddfd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:58.859640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:58.873003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5550ee9-81cf-4c41-95d4-179e791fddfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-b5550ee9-81cf-4c41-95d4-179e791fddfd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:58.887660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:58.888538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:58.889456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 200/403] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:45:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:45:58.894556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5550ee9-81cf-4c41-95d4-179e791fddfd None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:58.937856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dc6b5-bb8f-4e76-a7ad-1f636232ed3a None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:58.978605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a0e70-9363-4d56-a2a9-5a156085deec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:58.979765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a0e70-9363-4d56-a2a9-5a156085deec None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:58.980306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a0e70-9363-4d56-a2a9-5a156085deec None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:58.980718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a0e70-9363-4d56-a2a9-5a156085deec None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46/groups/d497ceffe4bf48c3aa012faf1295a02f/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:58.982848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a0e70-9363-4d56-a2a9-5a156085deec None admin] RBAC: Authorizing `identity:create_grant(project_id=197b0ba2357a405ca5849973bb209b46, group_id=d497ceffe4bf48c3aa012faf1295a02f, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:58.984423 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a0e70-9363-4d56-a2a9-5a156085deec None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:58.988085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:59.006714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 204/404] 10.222.0.45 () {58 vars in 1442 bytes} [Tue May 24 12:45:58 2022] PUT /identity/v3/projects/197b0ba2357a405ca5849973bb209b46/groups/d497ceffe4bf48c3aa012faf1295a02f/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 12:45:59.101330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5550ee9-81cf-4c41-95d4-179e791fddfd None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:59.176462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79de558-62aa-4568-8a60-90ddfcee5160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:59.177900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79de558-62aa-4568-8a60-90ddfcee5160 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:59.178522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79de558-62aa-4568-8a60-90ddfcee5160 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:59.179137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79de558-62aa-4568-8a60-90ddfcee5160 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:59.180460 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79de558-62aa-4568-8a60-90ddfcee5160 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:45:59.181101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79de558-62aa-4568-8a60-90ddfcee5160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:59.183326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79de558-62aa-4568-8a60-90ddfcee5160 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:59.197688 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 201/405] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:45:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:45:59.202966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a0e70-9363-4d56-a2a9-5a156085deec None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:59.322477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57936d-cf5b-4793-983d-8f21234a741b None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:59.323121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57936d-cf5b-4793-983d-8f21234a741b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:59.323613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57936d-cf5b-4793-983d-8f21234a741b None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:59.326213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 202/406] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:45:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:45:59.330470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0530bb8-7593-4f31-8f30-66ce714afcbb None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:59.331023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0530bb8-7593-4f31-8f30-66ce714afcbb None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:59.331499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0530bb8-7593-4f31-8f30-66ce714afcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:59.338558 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a0e70-9363-4d56-a2a9-5a156085deec None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:59.340136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:59.355155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e0530bb8-7593-4f31-8f30-66ce714afcbb None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:45:59.356731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e0530bb8-7593-4f31-8f30-66ce714afcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:45:59.381329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29434a-5c1a-41a8-9dc4-46fd78a1ca93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:59.381329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29434a-5c1a-41a8-9dc4-46fd78a1ca93 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:59.381329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29434a-5c1a-41a8-9dc4-46fd78a1ca93 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:59.381329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29434a-5c1a-41a8-9dc4-46fd78a1ca93 None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:59.382131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29434a-5c1a-41a8-9dc4-46fd78a1ca93 None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:59.385303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29434a-5c1a-41a8-9dc4-46fd78a1ca93 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:59.385590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 205/407] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:45:59 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:59.390997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29434a-5c1a-41a8-9dc4-46fd78a1ca93 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:59.511956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 203/408] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:45:59.546754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0530bb8-7593-4f31-8f30-66ce714afcbb None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:59.587997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29434a-5c1a-41a8-9dc4-46fd78a1ca93 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:59.634635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:59.684388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef264b-4d6f-47e6-8221-d62dec0abf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:59.685671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef264b-4d6f-47e6-8221-d62dec0abf02 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:59.686164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef264b-4d6f-47e6-8221-d62dec0abf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:59.686668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef264b-4d6f-47e6-8221-d62dec0abf02 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:59.693664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef264b-4d6f-47e6-8221-d62dec0abf02 None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:59.695291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef264b-4d6f-47e6-8221-d62dec0abf02 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:59.697402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 206/409] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:45:59 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:45:59.702562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef264b-4d6f-47e6-8221-d62dec0abf02 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:59.721941 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0530bb8-7593-4f31-8f30-66ce714afcbb None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:59.766983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1682ec-7dab-4793-92b3-a493fcdf41d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:59.768216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1682ec-7dab-4793-92b3-a493fcdf41d2 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:59.768760 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1682ec-7dab-4793-92b3-a493fcdf41d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:59.769224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1682ec-7dab-4793-92b3-a493fcdf41d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:59.789951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1682ec-7dab-4793-92b3-a493fcdf41d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:45:59.792991 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-ad1682ec-7dab-4793-92b3-a493fcdf41d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:45:59.800563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:45:59.801518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:45:59.802769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 204/410] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:45:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:45:59.809500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1682ec-7dab-4793-92b3-a493fcdf41d2 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:45:59.906208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef264b-4d6f-47e6-8221-d62dec0abf02 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:45:59.973647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40184110-febe-45ff-a7d9-e2477683eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:45:59.974668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40184110-febe-45ff-a7d9-e2477683eab2 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:45:59.974885 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40184110-febe-45ff-a7d9-e2477683eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:45:59.976696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40184110-febe-45ff-a7d9-e2477683eab2 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46/groups/d497ceffe4bf48c3aa012faf1295a02f/roles/c7b7d3b4dd944936851dee41ead5dc99` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:45:59.976834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:45:59.979600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40184110-febe-45ff-a7d9-e2477683eab2 None admin] RBAC: Authorizing `identity:create_grant(project_id=197b0ba2357a405ca5849973bb209b46, group_id=d497ceffe4bf48c3aa012faf1295a02f, role_id=c7b7d3b4dd944936851dee41ead5dc99)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:45:59.981129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40184110-febe-45ff-a7d9-e2477683eab2 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:45:59.989146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1682ec-7dab-4793-92b3-a493fcdf41d2 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:00.004173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 207/411] 10.222.0.45 () {58 vars in 1442 bytes} [Tue May 24 12:45:59 2022] PUT /identity/v3/projects/197b0ba2357a405ca5849973bb209b46/groups/d497ceffe4bf48c3aa012faf1295a02f/roles/c7b7d3b4dd944936851dee41ead5dc99 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 12:46:00.017719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98bd3-1928-44e4-b7c6-d3d470a04923 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:00.018057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98bd3-1928-44e4-b7c6-d3d470a04923 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:00.018641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98bd3-1928-44e4-b7c6-d3d470a04923 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:00.022201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 208/412] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:00.028693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80058f-b016-4b2a-9c48-5f9dbaa316d0 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:00.028982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80058f-b016-4b2a-9c48-5f9dbaa316d0 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:00.029311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80058f-b016-4b2a-9c48-5f9dbaa316d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:00.044856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0f1e9-b4b1-42b0-ad83-5caec01f9cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:00.046324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0f1e9-b4b1-42b0-ad83-5caec01f9cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:00.046818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0f1e9-b4b1-42b0-ad83-5caec01f9cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:00.047293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0f1e9-b4b1-42b0-ad83-5caec01f9cf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:00.048296 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0f1e9-b4b1-42b0-ad83-5caec01f9cf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:00.048835 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0f1e9-b4b1-42b0-ad83-5caec01f9cf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:00.050045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9b80058f-b016-4b2a-9c48-5f9dbaa316d0 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:00.050719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0f1e9-b4b1-42b0-ad83-5caec01f9cf7 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:00.051370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9b80058f-b016-4b2a-9c48-5f9dbaa316d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:00.062716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 205/413] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:45:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:00.074316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0f1e9-b4b1-42b0-ad83-5caec01f9cf7 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:00.221328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 209/414] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:46:00.245946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80058f-b016-4b2a-9c48-5f9dbaa316d0 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:00.280986 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:00.309303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0f1e9-b4b1-42b0-ad83-5caec01f9cf7 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:00.373581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80058f-b016-4b2a-9c48-5f9dbaa316d0 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:00.419044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40db057-42b5-4036-9931-c398e8c30017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:00.421066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db057-42b5-4036-9931-c398e8c30017 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:00.422117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db057-42b5-4036-9931-c398e8c30017 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:00.423348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db057-42b5-4036-9931-c398e8c30017 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:00.423682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1bfcb-6921-476a-8bc8-fa4db1d7c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:00.424799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1bfcb-6921-476a-8bc8-fa4db1d7c381 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:00.430814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1bfcb-6921-476a-8bc8-fa4db1d7c381 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:00.431119 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1bfcb-6921-476a-8bc8-fa4db1d7c381 None admin] PATH_INFO: `/v3/groups/5ddaebc8378a49fdb891f468fc7d8c4f` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:00.432517 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1bfcb-6921-476a-8bc8-fa4db1d7c381 None admin] RBAC: Authorizing `identity:get_group(group_id=5ddaebc8378a49fdb891f468fc7d8c4f)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:00.434030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1bfcb-6921-476a-8bc8-fa4db1d7c381 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:00.435893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 206/415] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:46:00 2022] GET /identity/v3/groups/5ddaebc8378a49fdb891f468fc7d8c4f => generated 229 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:00.443884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1bfcb-6921-476a-8bc8-fa4db1d7c381 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:00.445527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40db057-42b5-4036-9931-c398e8c30017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-a40db057-42b5-4036-9931-c398e8c30017 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:00.454046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:00.454985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:46:00.455790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:00.458529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 210/416] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:00.465942 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40db057-42b5-4036-9931-c398e8c30017 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:00.488342 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:00.576288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:00.593254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40db057-42b5-4036-9931-c398e8c30017 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:00.603822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1bfcb-6921-476a-8bc8-fa4db1d7c381 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:00.642836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2e98b-1736-45ba-a993-8315214397a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:00.643922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e98b-1736-45ba-a993-8315214397a0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:00.644435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e98b-1736-45ba-a993-8315214397a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:00.644913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e98b-1736-45ba-a993-8315214397a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:00.645879 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2e98b-1736-45ba-a993-8315214397a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:00.646422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2e98b-1736-45ba-a993-8315214397a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:00.648062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2e98b-1736-45ba-a993-8315214397a0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:00.656391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965df93-2429-4c8c-a984-c6ba67a3e309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:00.657335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965df93-2429-4c8c-a984-c6ba67a3e309 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:00.657624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965df93-2429-4c8c-a984-c6ba67a3e309 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:00.657905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965df93-2429-4c8c-a984-c6ba67a3e309 None admin] PATH_INFO: `/v3/projects/26aa7dae9c0c4f5a892a6fcef95c15ea` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:00.658953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 211/417] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:46:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:00.659908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965df93-2429-4c8c-a984-c6ba67a3e309 None admin] RBAC: Authorizing `identity:get_project(project_id=26aa7dae9c0c4f5a892a6fcef95c15ea)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:00.661572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965df93-2429-4c8c-a984-c6ba67a3e309 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:00.663886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2e98b-1736-45ba-a993-8315214397a0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:00.668308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 207/418] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:46:00 2022] GET /identity/v3/projects/26aa7dae9c0c4f5a892a6fcef95c15ea => generated 344 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:00.674614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965df93-2429-4c8c-a984-c6ba67a3e309 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:00.771692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:00.780743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965df93-2429-4c8c-a984-c6ba67a3e309 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:00.785391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2e98b-1736-45ba-a993-8315214397a0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:00.816402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc878467-8513-481f-b285-0a44e8baa3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:00.817520 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc878467-8513-481f-b285-0a44e8baa3d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:00.817702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc878467-8513-481f-b285-0a44e8baa3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:00.817950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc878467-8513-481f-b285-0a44e8baa3d4 None admin] PATH_INFO: `/v3/projects/26aa7dae9c0c4f5a892a6fcef95c15ea/groups/5ddaebc8378a49fdb891f468fc7d8c4f/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:00.820130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc878467-8513-481f-b285-0a44e8baa3d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=26aa7dae9c0c4f5a892a6fcef95c15ea, group_id=5ddaebc8378a49fdb891f468fc7d8c4f, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:00.821706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc878467-8513-481f-b285-0a44e8baa3d4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:00.826771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d0955-6be3-4b11-aa05-efa24b156712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:00.827779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d0955-6be3-4b11-aa05-efa24b156712 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:00.828428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d0955-6be3-4b11-aa05-efa24b156712 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:00.829369 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d0955-6be3-4b11-aa05-efa24b156712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:00.842656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 208/419] 10.222.0.45 () {58 vars in 1442 bytes} [Tue May 24 12:46:00 2022] PUT /identity/v3/projects/26aa7dae9c0c4f5a892a6fcef95c15ea/groups/5ddaebc8378a49fdb891f468fc7d8c4f/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 12:46:00.844901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b49b77-00bf-49da-ad0a-e5f001070845 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:00.845419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b49b77-00bf-49da-ad0a-e5f001070845 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:00.846009 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b49b77-00bf-49da-ad0a-e5f001070845 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:00.848101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 209/420] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:00.853744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:00.866567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f605291-de53-4822-89d1-960641bc527b None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:00.867201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f605291-de53-4822-89d1-960641bc527b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:00.867718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f605291-de53-4822-89d1-960641bc527b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:00.907484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9f605291-de53-4822-89d1-960641bc527b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:00.909107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9f605291-de53-4822-89d1-960641bc527b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:00.909216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d0955-6be3-4b11-aa05-efa24b156712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:00.910789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d0955-6be3-4b11-aa05-efa24b156712 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:01.047231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 212/421] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:46:01.053511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d0955-6be3-4b11-aa05-efa24b156712 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:01.107811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 210/422] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:46:01.116504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:01.144079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f605291-de53-4822-89d1-960641bc527b None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:01.182897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d0955-6be3-4b11-aa05-efa24b156712 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:01.234620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a2d96-575a-4285-b65d-3aecb60861fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:01.239407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2d96-575a-4285-b65d-3aecb60861fa None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:01.240268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2d96-575a-4285-b65d-3aecb60861fa None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:01.240556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2d96-575a-4285-b65d-3aecb60861fa None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:01.241925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a2d96-575a-4285-b65d-3aecb60861fa None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:01.243848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a2d96-575a-4285-b65d-3aecb60861fa None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:01.246826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 213/423] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:46:01 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:01.250124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:01.255550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a2d96-575a-4285-b65d-3aecb60861fa None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:01.289891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f605291-de53-4822-89d1-960641bc527b None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:01.353204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55161de1-1c02-4bf6-8fe1-7c7caf67acd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:01.354409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55161de1-1c02-4bf6-8fe1-7c7caf67acd4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:01.354892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55161de1-1c02-4bf6-8fe1-7c7caf67acd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:01.355388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55161de1-1c02-4bf6-8fe1-7c7caf67acd4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:01.395517 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55161de1-1c02-4bf6-8fe1-7c7caf67acd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-55161de1-1c02-4bf6-8fe1-7c7caf67acd4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:01.406191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:01.407219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:46:01.408491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:01.409107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 211/424] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:01.417328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55161de1-1c02-4bf6-8fe1-7c7caf67acd4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:01.433201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a2d96-575a-4285-b65d-3aecb60861fa None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:01.468235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:01.488978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392eb57a-aafb-4e71-ab8d-b489e31f5fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:01.489922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eb57a-aafb-4e71-ab8d-b489e31f5fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:01.490212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eb57a-aafb-4e71-ab8d-b489e31f5fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:01.490541 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eb57a-aafb-4e71-ab8d-b489e31f5fb7 None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:01.491843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392eb57a-aafb-4e71-ab8d-b489e31f5fb7 None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:01.496269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392eb57a-aafb-4e71-ab8d-b489e31f5fb7 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:01.496269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 214/425] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:46:01 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:01.501678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392eb57a-aafb-4e71-ab8d-b489e31f5fb7 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:01.622586 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55161de1-1c02-4bf6-8fe1-7c7caf67acd4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:01.667002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392eb57a-aafb-4e71-ab8d-b489e31f5fb7 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:01.703172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919ba42-9552-4ef9-b0e3-ca4376f05c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:01.704243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919ba42-9552-4ef9-b0e3-ca4376f05c70 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:01.704751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919ba42-9552-4ef9-b0e3-ca4376f05c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:01.705232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919ba42-9552-4ef9-b0e3-ca4376f05c70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:01.706267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919ba42-9552-4ef9-b0e3-ca4376f05c70 None admin] RBAC: Adding query filter params (group.id=d497ceffe4bf48c3aa012faf1295a02f, role.id=27d8d0ea0b83488a95a9141d439247f6, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:01.706822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919ba42-9552-4ef9-b0e3-ca4376f05c70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:01.707547 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686df33-202a-4799-b5b8-76ce8ed93174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:01.708231 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686df33-202a-4799-b5b8-76ce8ed93174 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:01.708517 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686df33-202a-4799-b5b8-76ce8ed93174 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:01.708797 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686df33-202a-4799-b5b8-76ce8ed93174 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:01.709376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919ba42-9552-4ef9-b0e3-ca4376f05c70 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:01.709565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686df33-202a-4799-b5b8-76ce8ed93174 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:01.709865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686df33-202a-4799-b5b8-76ce8ed93174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:01.711274 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686df33-202a-4799-b5b8-76ce8ed93174 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:01.716263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 215/426] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 12:46:01 2022] GET /identity/v3/role_assignments?group.id=d497ceffe4bf48c3aa012faf1295a02f&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:01.717418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aef363-74c8-4295-9d10-fcb953a6a71e None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:01.717676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aef363-74c8-4295-9d10-fcb953a6a71e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:01.717948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aef363-74c8-4295-9d10-fcb953a6a71e None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:01.719851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 216/427] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:01.730667 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 212/428] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:01.736347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aef363-74c8-4295-9d10-fcb953a6a71e None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:01.744341 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7d88-ae60-419c-890e-c0a327f1d901 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:01.745173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7d88-ae60-419c-890e-c0a327f1d901 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:01.745661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7d88-ae60-419c-890e-c0a327f1d901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:01.755810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:01.768451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9bfb7d88-ae60-419c-890e-c0a327f1d901 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:01.770114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9bfb7d88-ae60-419c-890e-c0a327f1d901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:01.823142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:01.876235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 213/429] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:01.895141 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aef363-74c8-4295-9d10-fcb953a6a71e None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:01.902081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7d88-ae60-419c-890e-c0a327f1d901 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:01.984802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a09a9f-c846-4b2b-ad31-a30a1c480bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:01.985787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a09a9f-c846-4b2b-ad31-a30a1c480bfc None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:01.986214 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a09a9f-c846-4b2b-ad31-a30a1c480bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:01.986651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a09a9f-c846-4b2b-ad31-a30a1c480bfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:02.086142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a09a9f-c846-4b2b-ad31-a30a1c480bfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:02.088765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a09a9f-c846-4b2b-ad31-a30a1c480bfc None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:02.091939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7d88-ae60-419c-890e-c0a327f1d901 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:02.151431 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac1cee-8f8c-4a69-8646-0f18839a2d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:02.152252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac1cee-8f8c-4a69-8646-0f18839a2d7e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:02.153389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac1cee-8f8c-4a69-8646-0f18839a2d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:02.153389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac1cee-8f8c-4a69-8646-0f18839a2d7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:02.155157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:02.156396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:02.174070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac1cee-8f8c-4a69-8646-0f18839a2d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-d8ac1cee-8f8c-4a69-8646-0f18839a2d7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:02.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:02.186324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 12:46:02.187151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:02.189621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 214/430] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:02.193662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac1cee-8f8c-4a69-8646-0f18839a2d7e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:02.253101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 217/431] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:02.259336 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a09a9f-c846-4b2b-ad31-a30a1c480bfc None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:02.323178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac1cee-8f8c-4a69-8646-0f18839a2d7e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:02.369121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238d98e-20f3-4553-916c-c5e25fd29eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:02.370107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238d98e-20f3-4553-916c-c5e25fd29eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:02.370653 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238d98e-20f3-4553-916c-c5e25fd29eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:02.371080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238d98e-20f3-4553-916c-c5e25fd29eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:02.372050 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238d98e-20f3-4553-916c-c5e25fd29eb9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:02.372527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238d98e-20f3-4553-916c-c5e25fd29eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:02.374230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238d98e-20f3-4553-916c-c5e25fd29eb9 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:02.380987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a09a9f-c846-4b2b-ad31-a30a1c480bfc None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:02.385026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 215/432] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:46:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:02.391639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238d98e-20f3-4553-916c-c5e25fd29eb9 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:02.419547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9e6a2-ef36-41ec-b49a-8a3c68baf7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:02.420424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e6a2-ef36-41ec-b49a-8a3c68baf7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:02.420711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e6a2-ef36-41ec-b49a-8a3c68baf7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:02.420990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e6a2-ef36-41ec-b49a-8a3c68baf7b9 None admin] PATH_INFO: `/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:02.422202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9e6a2-ef36-41ec-b49a-8a3c68baf7b9 None admin] RBAC: Authorizing `identity:get_project(project_id=a75c6b7e46c24ae8adacf3d5669e7162)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:02.424250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9e6a2-ef36-41ec-b49a-8a3c68baf7b9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:02.430451 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 218/433] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:46:02 2022] GET /identity/v3/projects/a75c6b7e46c24ae8adacf3d5669e7162 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:02.431289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9e6a2-ef36-41ec-b49a-8a3c68baf7b9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:02.450946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:02.458538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:02.507682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238d98e-20f3-4553-916c-c5e25fd29eb9 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:02.541844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9e6a2-ef36-41ec-b49a-8a3c68baf7b9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:02.545028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fecff-42ad-4579-b318-b5c27747d4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:02.545838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fecff-42ad-4579-b318-b5c27747d4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:02.546264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fecff-42ad-4579-b318-b5c27747d4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:02.546706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fecff-42ad-4579-b318-b5c27747d4d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:02.578952 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46bf34-a280-4926-97af-4f1126299bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:02.579930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46bf34-a280-4926-97af-4f1126299bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:02.580225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46bf34-a280-4926-97af-4f1126299bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:02.580511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46bf34-a280-4926-97af-4f1126299bb8 None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:02.581888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46bf34-a280-4926-97af-4f1126299bb8 None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:02.583942 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46bf34-a280-4926-97af-4f1126299bb8 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:02.586154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 219/434] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:46:02 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:02.591768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46bf34-a280-4926-97af-4f1126299bb8 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:02.621736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fecff-42ad-4579-b318-b5c27747d4d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:02.623465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fecff-42ad-4579-b318-b5c27747d4d4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:02.682737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:02.691969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:02.704809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46bf34-a280-4926-97af-4f1126299bb8 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:02.742499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 216/435] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:46:02.745592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e465588-ca87-42a9-9915-118823039d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:02.746253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e465588-ca87-42a9-9915-118823039d31 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:02.746568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e465588-ca87-42a9-9915-118823039d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:02.746973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e465588-ca87-42a9-9915-118823039d31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:02.747772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e465588-ca87-42a9-9915-118823039d31 None admin] RBAC: Adding query filter params (group.id=d497ceffe4bf48c3aa012faf1295a02f, role.id=c7b7d3b4dd944936851dee41ead5dc99, scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:02.748091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e465588-ca87-42a9-9915-118823039d31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:02.748183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb9d02-48fe-46d2-9691-2af1d3ee9f07 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:02.748426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb9d02-48fe-46d2-9691-2af1d3ee9f07 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:02.748699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb9d02-48fe-46d2-9691-2af1d3ee9f07 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:02.749534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e465588-ca87-42a9-9915-118823039d31 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:02.749965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 217/436] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:02.751168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb9d02-48fe-46d2-9691-2af1d3ee9f07 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:02.757127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 220/437] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 12:46:02 2022] GET /identity/v3/role_assignments?group.id=d497ceffe4bf48c3aa012faf1295a02f&scope.project.id=a75c6b7e46c24ae8adacf3d5669e7162&role.id=c7b7d3b4dd944936851dee41ead5dc99 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:02.760326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74722435-0d60-4d56-89cf-9238509409fc None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:02.760850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74722435-0d60-4d56-89cf-9238509409fc None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:02.761593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74722435-0d60-4d56-89cf-9238509409fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:02.788135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-74722435-0d60-4d56-89cf-9238509409fc None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:02.790115 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-74722435-0d60-4d56-89cf-9238509409fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:02.867604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb9d02-48fe-46d2-9691-2af1d3ee9f07 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:02.893760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 221/438] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:02.911691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18f1bc-0137-4a1c-9b37-c5a26d794210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:02.912893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18f1bc-0137-4a1c-9b37-c5a26d794210 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:02.913451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18f1bc-0137-4a1c-9b37-c5a26d794210 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:02.913871 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18f1bc-0137-4a1c-9b37-c5a26d794210 None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:02.915418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18f1bc-0137-4a1c-9b37-c5a26d794210 None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:02.917094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18f1bc-0137-4a1c-9b37-c5a26d794210 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:02.919530 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 218/439] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:46:02 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:02.925827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18f1bc-0137-4a1c-9b37-c5a26d794210 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:02.926903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74722435-0d60-4d56-89cf-9238509409fc None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:02.956675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:03.033655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:03.055016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18f1bc-0137-4a1c-9b37-c5a26d794210 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:03.056491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74722435-0d60-4d56-89cf-9238509409fc None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:03.106986 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2486ef-95a4-4b74-866d-aef966fd2c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:03.108694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2486ef-95a4-4b74-866d-aef966fd2c6b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:03.108929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d76592-895c-41df-85d2-10f6f9f6dba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:03.109291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2486ef-95a4-4b74-866d-aef966fd2c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:03.110246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2486ef-95a4-4b74-866d-aef966fd2c6b None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:03.110413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d76592-895c-41df-85d2-10f6f9f6dba0 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:03.110681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d76592-895c-41df-85d2-10f6f9f6dba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:03.110955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d76592-895c-41df-85d2-10f6f9f6dba0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:03.112181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2486ef-95a4-4b74-866d-aef966fd2c6b None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:03.113941 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2486ef-95a4-4b74-866d-aef966fd2c6b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:03.116959 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 222/440] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:46:02 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:03.122902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2486ef-95a4-4b74-866d-aef966fd2c6b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:03.123129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d76592-895c-41df-85d2-10f6f9f6dba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-40d76592-895c-41df-85d2-10f6f9f6dba0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:03.133063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:03.133995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:03.134845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 219/441] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:03.140322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d76592-895c-41df-85d2-10f6f9f6dba0 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:03.245140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:03.253421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2486ef-95a4-4b74-866d-aef966fd2c6b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:03.263443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d76592-895c-41df-85d2-10f6f9f6dba0 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:03.290770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55b2f1-2879-4d4b-9df3-1a30d5df7bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:03.291442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55b2f1-2879-4d4b-9df3-1a30d5df7bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:03.291726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55b2f1-2879-4d4b-9df3-1a30d5df7bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:03.292006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55b2f1-2879-4d4b-9df3-1a30d5df7bf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:03.292845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55b2f1-2879-4d4b-9df3-1a30d5df7bf9 None admin] RBAC: Adding query filter params (group.id=d497ceffe4bf48c3aa012faf1295a02f, role.id=27d8d0ea0b83488a95a9141d439247f6, scope.project.id=197b0ba2357a405ca5849973bb209b46) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:03.293159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55b2f1-2879-4d4b-9df3-1a30d5df7bf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:03.294627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55b2f1-2879-4d4b-9df3-1a30d5df7bf9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:03.299058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31679b1-e2f1-437b-bd9c-f01cb69f209e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:03.299732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31679b1-e2f1-437b-bd9c-f01cb69f209e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:03.300018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31679b1-e2f1-437b-bd9c-f01cb69f209e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:03.300295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31679b1-e2f1-437b-bd9c-f01cb69f209e None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:03.301049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31679b1-e2f1-437b-bd9c-f01cb69f209e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:03.301542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31679b1-e2f1-437b-bd9c-f01cb69f209e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:03.301640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 223/442] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 12:46:03 2022] GET /identity/v3/role_assignments?group.id=d497ceffe4bf48c3aa012faf1295a02f&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=27d8d0ea0b83488a95a9141d439247f6 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:03.302925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31679b1-e2f1-437b-bd9c-f01cb69f209e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:03.312204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 220/443] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:03.317995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55b2f1-2879-4d4b-9df3-1a30d5df7bf9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:03.334742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:03.443000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55b2f1-2879-4d4b-9df3-1a30d5df7bf9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:03.479728 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8a1d3-b1f1-4c9b-9134-d50addac17d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:03.480399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8a1d3-b1f1-4c9b-9134-d50addac17d8 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:03.480687 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8a1d3-b1f1-4c9b-9134-d50addac17d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:03.480967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8a1d3-b1f1-4c9b-9134-d50addac17d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:03.555657 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8a1d3-b1f1-4c9b-9134-d50addac17d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:03.557042 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8a1d3-b1f1-4c9b-9134-d50addac17d8 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:03.571914 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:03.580311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cdf7f-8724-4a2b-b749-6e0dbdb23346 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:03.580603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cdf7f-8724-4a2b-b749-6e0dbdb23346 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:03.580911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cdf7f-8724-4a2b-b749-6e0dbdb23346 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:03.582256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 221/444] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:03.587981 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308aee-98bf-4f60-884f-c3e341f2ba25 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:03.588269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308aee-98bf-4f60-884f-c3e341f2ba25 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:03.588544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308aee-98bf-4f60-884f-c3e341f2ba25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:03.607256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-77308aee-98bf-4f60-884f-c3e341f2ba25 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:03.609242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-77308aee-98bf-4f60-884f-c3e341f2ba25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:03.671225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 224/445] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:46:03.677584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8a1d3-b1f1-4c9b-9134-d50addac17d8 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:03.689962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 222/446] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:46:03.713395 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308aee-98bf-4f60-884f-c3e341f2ba25 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:03.779213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8a1d3-b1f1-4c9b-9134-d50addac17d8 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:03.792795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:03.792946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:03.813843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ddc3b-7899-40de-ba4a-e215fc895c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:03.814723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ddc3b-7899-40de-ba4a-e215fc895c8b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:03.815005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ddc3b-7899-40de-ba4a-e215fc895c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:03.815280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ddc3b-7899-40de-ba4a-e215fc895c8b None admin] PATH_INFO: `/v3/projects/197b0ba2357a405ca5849973bb209b46` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:03.815645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308aee-98bf-4f60-884f-c3e341f2ba25 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:03.816480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ddc3b-7899-40de-ba4a-e215fc895c8b None admin] RBAC: Authorizing `identity:get_project(project_id=197b0ba2357a405ca5849973bb209b46)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:03.818007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ddc3b-7899-40de-ba4a-e215fc895c8b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:03.819706 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 225/447] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:46:03 2022] GET /identity/v3/projects/197b0ba2357a405ca5849973bb209b46 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:03.824976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ddc3b-7899-40de-ba4a-e215fc895c8b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:03.849590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6f369-4ec0-44aa-a307-2b7fdd000163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:03.850399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6f369-4ec0-44aa-a307-2b7fdd000163 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:03.850683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6f369-4ec0-44aa-a307-2b7fdd000163 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:03.850959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6f369-4ec0-44aa-a307-2b7fdd000163 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:03.860707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6f369-4ec0-44aa-a307-2b7fdd000163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-91e6f369-4ec0-44aa-a307-2b7fdd000163 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:03.868338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:03.869267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:03.870466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 223/448] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:03.875075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6f369-4ec0-44aa-a307-2b7fdd000163 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:03.922896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ddc3b-7899-40de-ba4a-e215fc895c8b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:03.956126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9dfcc-dbb5-4e5c-8016-15a5888f485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:03.956982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9dfcc-dbb5-4e5c-8016-15a5888f485c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:03.957263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9dfcc-dbb5-4e5c-8016-15a5888f485c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:03.957544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9dfcc-dbb5-4e5c-8016-15a5888f485c None admin] PATH_INFO: `/v3/groups/d497ceffe4bf48c3aa012faf1295a02f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:03.958741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9dfcc-dbb5-4e5c-8016-15a5888f485c None admin] RBAC: Authorizing `identity:get_group(group_id=d497ceffe4bf48c3aa012faf1295a02f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:03.960171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9dfcc-dbb5-4e5c-8016-15a5888f485c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:03.961827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 226/449] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:46:03 2022] GET /identity/v3/groups/d497ceffe4bf48c3aa012faf1295a02f => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:03.967649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9dfcc-dbb5-4e5c-8016-15a5888f485c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:03.972305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6f369-4ec0-44aa-a307-2b7fdd000163 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:04.005777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886c019-59c8-43b2-a12c-997ce5bb0c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:04.006484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886c019-59c8-43b2-a12c-997ce5bb0c4b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:04.006772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886c019-59c8-43b2-a12c-997ce5bb0c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:04.007050 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886c019-59c8-43b2-a12c-997ce5bb0c4b None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:04.007792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886c019-59c8-43b2-a12c-997ce5bb0c4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:04.008098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886c019-59c8-43b2-a12c-997ce5bb0c4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:04.009468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886c019-59c8-43b2-a12c-997ce5bb0c4b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:04.012258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:04.015157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:04.019749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 224/450] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:04.024985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886c019-59c8-43b2-a12c-997ce5bb0c4b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:04.070824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9dfcc-dbb5-4e5c-8016-15a5888f485c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:04.104350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4223d1-c88a-4abf-bfd1-0f234bf41827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:04.105007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4223d1-c88a-4abf-bfd1-0f234bf41827 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:04.105293 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4223d1-c88a-4abf-bfd1-0f234bf41827 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:04.105569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4223d1-c88a-4abf-bfd1-0f234bf41827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:04.106430 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4223d1-c88a-4abf-bfd1-0f234bf41827 None admin] RBAC: Adding query filter params (group.id=d497ceffe4bf48c3aa012faf1295a02f, role.id=c7b7d3b4dd944936851dee41ead5dc99, scope.project.id=197b0ba2357a405ca5849973bb209b46) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:04.106742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4223d1-c88a-4abf-bfd1-0f234bf41827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:04.108181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4223d1-c88a-4abf-bfd1-0f234bf41827 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:04.114351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 227/451] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 12:46:03 2022] GET /identity/v3/role_assignments?group.id=d497ceffe4bf48c3aa012faf1295a02f&scope.project.id=197b0ba2357a405ca5849973bb209b46&role.id=c7b7d3b4dd944936851dee41ead5dc99 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:04.123007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886c019-59c8-43b2-a12c-997ce5bb0c4b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:04.159718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cdb33-cc17-4723-a7af-155305b1d039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:04.160382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cdb33-cc17-4723-a7af-155305b1d039 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:04.160689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cdb33-cc17-4723-a7af-155305b1d039 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:04.160951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cdb33-cc17-4723-a7af-155305b1d039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:04.229287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:04.234011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cdb33-cc17-4723-a7af-155305b1d039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:04.236130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cdb33-cc17-4723-a7af-155305b1d039 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:04.340538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 225/452] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:46:04.346860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4223d1-c88a-4abf-bfd1-0f234bf41827 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:04.457004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4223d1-c88a-4abf-bfd1-0f234bf41827 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:04.471603 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:04.492870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4016ec-8419-4248-8aa0-cccb62891207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:04.493736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4016ec-8419-4248-8aa0-cccb62891207 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:04.494024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4016ec-8419-4248-8aa0-cccb62891207 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:04.494359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4016ec-8419-4248-8aa0-cccb62891207 None admin] PATH_INFO: `/v3/projects/26aa7dae9c0c4f5a892a6fcef95c15ea` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:04.495532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4016ec-8419-4248-8aa0-cccb62891207 None admin] RBAC: Authorizing `identity:get_project(project_id=26aa7dae9c0c4f5a892a6fcef95c15ea)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:04.496952 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4016ec-8419-4248-8aa0-cccb62891207 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:04.498650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 228/453] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:46:04 2022] GET /identity/v3/projects/26aa7dae9c0c4f5a892a6fcef95c15ea => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:04.504146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cdb33-cc17-4723-a7af-155305b1d039 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:04.597534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:04.601353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cdb33-cc17-4723-a7af-155305b1d039 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:04.634264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b5e00-dea2-4f11-806d-010045abf0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:04.635125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b5e00-dea2-4f11-806d-010045abf0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:04.635404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b5e00-dea2-4f11-806d-010045abf0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:04.635678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b5e00-dea2-4f11-806d-010045abf0c1 None admin] PATH_INFO: `/v3/groups/5ddaebc8378a49fdb891f468fc7d8c4f` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:04.637077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b5e00-dea2-4f11-806d-010045abf0c1 None admin] RBAC: Authorizing `identity:get_group(group_id=5ddaebc8378a49fdb891f468fc7d8c4f)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:04.638549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b5e00-dea2-4f11-806d-010045abf0c1 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:04.640150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 226/454] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 12:46:04 2022] GET /identity/v3/groups/5ddaebc8378a49fdb891f468fc7d8c4f => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:04.646452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4016ec-8419-4248-8aa0-cccb62891207 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:04.746054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4016ec-8419-4248-8aa0-cccb62891207 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:04.784854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85ea5e-55d0-47f3-b07f-cc68c1fa248c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:04.785498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ea5e-55d0-47f3-b07f-cc68c1fa248c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:04.785778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ea5e-55d0-47f3-b07f-cc68c1fa248c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:04.786054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ea5e-55d0-47f3-b07f-cc68c1fa248c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:04.786898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85ea5e-55d0-47f3-b07f-cc68c1fa248c None admin] RBAC: Adding query filter params (group.id=5ddaebc8378a49fdb891f468fc7d8c4f, role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=26aa7dae9c0c4f5a892a6fcef95c15ea) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:04.787207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85ea5e-55d0-47f3-b07f-cc68c1fa248c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:04.788637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85ea5e-55d0-47f3-b07f-cc68c1fa248c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:04.794532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 229/455] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 12:46:04 2022] GET /identity/v3/role_assignments?group.id=5ddaebc8378a49fdb891f468fc7d8c4f&scope.project.id=26aa7dae9c0c4f5a892a6fcef95c15ea&role.id=0c473678b281495f86f58b077be04e1b => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:07.595682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855ec5-3534-4623-addf-a8bbfdfcd235 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:07.596190 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855ec5-3534-4623-addf-a8bbfdfcd235 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:07.596265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855ec5-3534-4623-addf-a8bbfdfcd235 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:07.597838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 227/456] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:07.612548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f08c10-046e-4620-b9d6-e19518ba3e5d None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:07.613130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f08c10-046e-4620-b9d6-e19518ba3e5d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:07.613130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f08c10-046e-4620-b9d6-e19518ba3e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:07.639276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e6f08c10-046e-4620-b9d6-e19518ba3e5d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:07.640471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e6f08c10-046e-4620-b9d6-e19518ba3e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:07.686617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:07.690729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb137b5-148b-4039-9ae3-661458f3afb9 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:07.691023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb137b5-148b-4039-9ae3-661458f3afb9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:07.691301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb137b5-148b-4039-9ae3-661458f3afb9 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:07.692917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 228/457] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:07.699006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c960b8-aa52-4890-8cd0-ad491883cd16 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:07.699296 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c960b8-aa52-4890-8cd0-ad491883cd16 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:07.699577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c960b8-aa52-4890-8cd0-ad491883cd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:07.724875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-15c960b8-aa52-4890-8cd0-ad491883cd16 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:07.726422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-15c960b8-aa52-4890-8cd0-ad491883cd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:07.737794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 230/458] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:07.789835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f08c10-046e-4620-b9d6-e19518ba3e5d None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:07.817735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 229/459] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:07.822303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120e5f0-df32-4deb-810b-c4df9eafb730 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:07.822303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120e5f0-df32-4deb-810b-c4df9eafb730 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:07.822303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120e5f0-df32-4deb-810b-c4df9eafb730 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:07.822303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 230/460] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:07.826487 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf332f4-6abc-479e-8086-3c7479725c42 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:07.826781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf332f4-6abc-479e-8086-3c7479725c42 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:07.827061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf332f4-6abc-479e-8086-3c7479725c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:07.848273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ddf332f4-6abc-479e-8086-3c7479725c42 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:07.849445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ddf332f4-6abc-479e-8086-3c7479725c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:07.932084 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 231/461] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:07.933374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf332f4-6abc-479e-8086-3c7479725c42 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:07.944444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f08c10-046e-4620-b9d6-e19518ba3e5d None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:07.967187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:08.001470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667168d-5201-4aae-a111-172fd0fad91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:08.003099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667168d-5201-4aae-a111-172fd0fad91a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.003765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667168d-5201-4aae-a111-172fd0fad91a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.004181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667168d-5201-4aae-a111-172fd0fad91a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.018944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667168d-5201-4aae-a111-172fd0fad91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:08.020856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667168d-5201-4aae-a111-172fd0fad91a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-a667168d-5201-4aae-a111-172fd0fad91a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:08.043677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:08.044757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:08.045816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:08.045816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:08.045816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:08.045816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:08.047503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 231/462] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:08.052282 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667168d-5201-4aae-a111-172fd0fad91a None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:08.074475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf332f4-6abc-479e-8086-3c7479725c42 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:08.079360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:08.123382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90263021-4adf-459b-a100-18b52c4563af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:08.125034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90263021-4adf-459b-a100-18b52c4563af None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.125381 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90263021-4adf-459b-a100-18b52c4563af None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.125660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90263021-4adf-459b-a100-18b52c4563af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.139064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90263021-4adf-459b-a100-18b52c4563af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:08.140782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90263021-4adf-459b-a100-18b52c4563af None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:08.154123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667168d-5201-4aae-a111-172fd0fad91a None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-90263021-4adf-459b-a100-18b52c4563af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:08.163647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:08.164482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:08.165310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:08.165310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:08.165310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:08.165310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:08.166130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 232/463] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:08.166996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90263021-4adf-459b-a100-18b52c4563af None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:08.194438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae85af5-a15e-40cf-8c64-3a92a7784680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:08.195298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae85af5-a15e-40cf-8c64-3a92a7784680 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.195533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae85af5-a15e-40cf-8c64-3a92a7784680 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.195821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae85af5-a15e-40cf-8c64-3a92a7784680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.207141 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae85af5-a15e-40cf-8c64-3a92a7784680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:08.208605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae85af5-a15e-40cf-8c64-3a92a7784680 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-cae85af5-a15e-40cf-8c64-3a92a7784680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:08.227149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:08.228027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:08.228909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:08.228909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:08.228909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:08.228909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:08.228909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 232/464] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:08.229885 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc3547-404d-4f07-b142-3ed90e939f3f None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.230173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc3547-404d-4f07-b142-3ed90e939f3f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.230478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc3547-404d-4f07-b142-3ed90e939f3f None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.231881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 233/465] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:08.236435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc3547-404d-4f07-b142-3ed90e939f3f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:08.254349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:08.279063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90263021-4adf-459b-a100-18b52c4563af None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:08.315458 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38d8c5-1fc2-4d3a-a5a6-a6e011e130fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:08.316240 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38d8c5-1fc2-4d3a-a5a6-a6e011e130fb None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.316470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38d8c5-1fc2-4d3a-a5a6-a6e011e130fb None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.316751 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38d8c5-1fc2-4d3a-a5a6-a6e011e130fb None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.317480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38d8c5-1fc2-4d3a-a5a6-a6e011e130fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:08.317790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38d8c5-1fc2-4d3a-a5a6-a6e011e130fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:08.319196 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38d8c5-1fc2-4d3a-a5a6-a6e011e130fb None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:08.320371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:08.337814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 233/466] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:08.337814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38d8c5-1fc2-4d3a-a5a6-a6e011e130fb None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:08.349457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc3547-404d-4f07-b142-3ed90e939f3f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:08.389789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e33ec-b0fa-48a6-b6b2-f352d4850bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:08.390491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e33ec-b0fa-48a6-b6b2-f352d4850bfe None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.390779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e33ec-b0fa-48a6-b6b2-f352d4850bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.391663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e33ec-b0fa-48a6-b6b2-f352d4850bfe None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.392299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e33ec-b0fa-48a6-b6b2-f352d4850bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:08.392609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e33ec-b0fa-48a6-b6b2-f352d4850bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:08.393998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e33ec-b0fa-48a6-b6b2-f352d4850bfe None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:08.409212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 234/467] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:08.410531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c249-c0d4-47d9-808d-f47c5a5ec138 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.410823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c249-c0d4-47d9-808d-f47c5a5ec138 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.411104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c249-c0d4-47d9-808d-f47c5a5ec138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.428807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-27d9c249-c0d4-47d9-808d-f47c5a5ec138 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:08.429998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-27d9c249-c0d4-47d9-808d-f47c5a5ec138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:08.457790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38d8c5-1fc2-4d3a-a5a6-a6e011e130fb None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:08.492555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9522acd-1816-4f09-b95c-be43004eacc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:08.493222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9522acd-1816-4f09-b95c-be43004eacc3 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.493504 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9522acd-1816-4f09-b95c-be43004eacc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.493783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9522acd-1816-4f09-b95c-be43004eacc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.494537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9522acd-1816-4f09-b95c-be43004eacc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:08.494849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9522acd-1816-4f09-b95c-be43004eacc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:08.496235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9522acd-1816-4f09-b95c-be43004eacc3 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:08.502533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:08.503107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 235/468] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:08.504072 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9c249-c0d4-47d9-808d-f47c5a5ec138 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:08.518881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 234/469] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:08.519665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9522acd-1816-4f09-b95c-be43004eacc3 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:08.584782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:08.623818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9c249-c0d4-47d9-808d-f47c5a5ec138 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:08.628825 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9522acd-1816-4f09-b95c-be43004eacc3 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:08.667407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b4de9-dd76-4f9e-8830-a2e914d878c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:08.667761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5db4a-0eae-4dae-a3a0-76d56829d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:08.668306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b4de9-dd76-4f9e-8830-a2e914d878c3 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.668493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5db4a-0eae-4dae-a3a0-76d56829d47d None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.668738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b4de9-dd76-4f9e-8830-a2e914d878c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.668835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5db4a-0eae-4dae-a3a0-76d56829d47d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.669044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5db4a-0eae-4dae-a3a0-76d56829d47d None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.669139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b4de9-dd76-4f9e-8830-a2e914d878c3 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.669804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5db4a-0eae-4dae-a3a0-76d56829d47d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:08.669903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b4de9-dd76-4f9e-8830-a2e914d878c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:08.671332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5db4a-0eae-4dae-a3a0-76d56829d47d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:08.671484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b4de9-dd76-4f9e-8830-a2e914d878c3 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:08.695253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 235/470] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:46:08 2022] POST /identity/v3/users => generated 248 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:46:08.696184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b4de9-dd76-4f9e-8830-a2e914d878c3 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:08.714399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 236/471] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:46:08 2022] POST /identity/v3/users => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:46:08.714399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5db4a-0eae-4dae-a3a0-76d56829d47d None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:08.809721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:08.889597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b4de9-dd76-4f9e-8830-a2e914d878c3 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:08.917028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5db4a-0eae-4dae-a3a0-76d56829d47d None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:08.948632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:08.964946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb810b-17d4-414e-8d38-62f66cf26c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:08.965912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb810b-17d4-414e-8d38-62f66cf26c51 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.966412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb810b-17d4-414e-8d38-62f66cf26c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.966689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb810b-17d4-414e-8d38-62f66cf26c51 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.967860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb810b-17d4-414e-8d38-62f66cf26c51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:08.968865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb810b-17d4-414e-8d38-62f66cf26c51 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:08.969266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7c597-41da-40c1-a57a-bea53019a74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:08.972457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7c597-41da-40c1-a57a-bea53019a74a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:08.972457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7c597-41da-40c1-a57a-bea53019a74a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:08.972457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7c597-41da-40c1-a57a-bea53019a74a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:08.993421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7c597-41da-40c1-a57a-bea53019a74a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:08.997071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7c597-41da-40c1-a57a-bea53019a74a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:09.008918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 236/472] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:46:08 2022] POST /identity/v3/users => generated 248 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:46:09.010728 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263da090-f155-4936-b128-855de8bd1ac3 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:09.011483 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263da090-f155-4936-b128-855de8bd1ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:09.012369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263da090-f155-4936-b128-855de8bd1ac3 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:09.013898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 237/473] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:09.024156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb64e4-6da2-496d-8bdc-3025a41845f2 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:09.024988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb64e4-6da2-496d-8bdc-3025a41845f2 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:09.024988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb64e4-6da2-496d-8bdc-3025a41845f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-c0c7c597-41da-40c1-a57a-bea53019a74a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:09.035243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:09.036321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:09.037386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:09.037386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:09.037386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:09.037386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:09.038672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 237/474] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:09.044290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7c597-41da-40c1-a57a-bea53019a74a None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:09.074068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-3ebb64e4-6da2-496d-8bdc-3025a41845f2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:09.077091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3ebb64e4-6da2-496d-8bdc-3025a41845f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:09.213058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 238/475] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:09.236499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb64e4-6da2-496d-8bdc-3025a41845f2 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:09.283081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:09.312227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7c597-41da-40c1-a57a-bea53019a74a None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:09.377017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5828e6-f661-44dd-bd5c-be2c16415836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:09.382043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5828e6-f661-44dd-bd5c-be2c16415836 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:09.382043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5828e6-f661-44dd-bd5c-be2c16415836 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:09.382043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5828e6-f661-44dd-bd5c-be2c16415836 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:09.382043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5828e6-f661-44dd-bd5c-be2c16415836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:09.382043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5828e6-f661-44dd-bd5c-be2c16415836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:09.382043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5828e6-f661-44dd-bd5c-be2c16415836 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:09.402276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 238/476] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:09.409580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5828e6-f661-44dd-bd5c-be2c16415836 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:09.426421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb64e4-6da2-496d-8bdc-3025a41845f2 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:09.465306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:09.474498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af07e-66d5-4466-a95f-a84e9f692f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:09.475186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af07e-66d5-4466-a95f-a84e9f692f53 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:09.475469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af07e-66d5-4466-a95f-a84e9f692f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:09.475755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af07e-66d5-4466-a95f-a84e9f692f53 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:09.476527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37af07e-66d5-4466-a95f-a84e9f692f53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:09.477911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37af07e-66d5-4466-a95f-a84e9f692f53 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:09.486053 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.sql.core [None req-f37af07e-66d5-4466-a95f-a84e9f692f53 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 12:46:09.486053 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 24 12:46:09.486053 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [parameters: {'id': '82be064f5d244d40832333d4598913ed', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 24 12:46:09.486053 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80090) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-f37af07e-66d5-4466-a95f-a84e9f692f53 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application result = self._query(query) May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application conn.query(q) May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application result.read() May 24 12:46:09.507610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return ref.to_dict() May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 12:46:09.508485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application next(self.gen) May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application yield resource May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application next(self.gen) May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self.session.rollback() May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self.force_reraise() May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise self.value May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application session.commit() May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 12:46:09.509992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._prepare_impl() May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self.session.flush() May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._flush(objects) May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application compat.raise_( May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application flush_context.execute() May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rec.execute(self) May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 12:46:09.511676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application c = connection._execute_20( May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 12:46:09.513370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application result = self._query(query) May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application conn.query(q) May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application result.read() May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 12:46:09.520043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [parameters: {'id': '82be064f5d244d40832333d4598913ed', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:09.521943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 24 12:46:09.523872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:09.525771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 239/477] 10.222.0.45 () {60 vars in 1131 bytes} [Tue May 24 12:46:09 2022] POST /identity/v3/roles => generated 144 bytes in 273 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:09.525771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af07e-66d5-4466-a95f-a84e9f692f53 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:09.611821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5828e6-f661-44dd-bd5c-be2c16415836 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:09.632770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af07e-66d5-4466-a95f-a84e9f692f53 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:09.643866 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:09.675278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe4a3b-df55-4cf7-8f7d-d5c133c1bd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:09.676364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe4a3b-df55-4cf7-8f7d-d5c133c1bd92 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:09.676652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe4a3b-df55-4cf7-8f7d-d5c133c1bd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:09.676956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe4a3b-df55-4cf7-8f7d-d5c133c1bd92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:09.691317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe4a3b-df55-4cf7-8f7d-d5c133c1bd92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:09.694574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00df5e-d443-4ac9-84a2-853e2700b44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:09.695260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00df5e-d443-4ac9-84a2-853e2700b44d None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:09.695544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00df5e-d443-4ac9-84a2-853e2700b44d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:09.695821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00df5e-d443-4ac9-84a2-853e2700b44d None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:09.696537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00df5e-d443-4ac9-84a2-853e2700b44d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-0cbe4a3b-df55-4cf7-8f7d-d5c133c1bd92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:09.700199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:09.702260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:09.704811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 240/478] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:09.710308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00df5e-d443-4ac9-84a2-853e2700b44d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:09.710555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe4a3b-df55-4cf7-8f7d-d5c133c1bd92 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:09.772338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 239/479] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:46:09 2022] POST /identity/v3/users => generated 249 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:46:09.827648 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe4a3b-df55-4cf7-8f7d-d5c133c1bd92 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:09.865979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4c082-517d-4ac8-ba4c-08bd6dbbf340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:09.866695 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4c082-517d-4ac8-ba4c-08bd6dbbf340 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:09.867152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4c082-517d-4ac8-ba4c-08bd6dbbf340 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:09.867491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4c082-517d-4ac8-ba4c-08bd6dbbf340 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:09.868249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4c082-517d-4ac8-ba4c-08bd6dbbf340 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:09.868557 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4c082-517d-4ac8-ba4c-08bd6dbbf340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:09.869951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4c082-517d-4ac8-ba4c-08bd6dbbf340 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:09.879065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 241/480] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:11.285000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046edaf6-cf00-4a13-8437-d631f4936c24 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:11.286110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046edaf6-cf00-4a13-8437-d631f4936c24 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:11.286624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046edaf6-cf00-4a13-8437-d631f4936c24 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:11.288428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 240/481] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:11.294040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc467e41-ac2e-4133-b7e5-fc3c824e4fc8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:11.294854 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc467e41-ac2e-4133-b7e5-fc3c824e4fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:11.295360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc467e41-ac2e-4133-b7e5-fc3c824e4fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:11.313321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-bc467e41-ac2e-4133-b7e5-fc3c824e4fc8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:11.314847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bc467e41-ac2e-4133-b7e5-fc3c824e4fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:11.337033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:11.392289 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 242/482] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:46:11.417775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046edaf6-cf00-4a13-8437-d631f4936c24 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:11.516739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:11.530958 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046edaf6-cf00-4a13-8437-d631f4936c24 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:11.570010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87296a-e075-4d15-a9c6-8e486584b6fb None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:11.570371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87296a-e075-4d15-a9c6-8e486584b6fb None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:11.570598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87296a-e075-4d15-a9c6-8e486584b6fb None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:11.571968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 243/483] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:11.576095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833eee5-b19c-41e4-8da9-f649fc651ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:11.576993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833eee5-b19c-41e4-8da9-f649fc651ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:11.577279 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833eee5-b19c-41e4-8da9-f649fc651ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:11.577557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833eee5-b19c-41e4-8da9-f649fc651ae2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:11.578622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c0fa-71c2-416d-ad83-ee987dd6a3de None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:11.579117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c0fa-71c2-416d-ad83-ee987dd6a3de None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:11.579592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c0fa-71c2-416d-ad83-ee987dd6a3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:11.588776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833eee5-b19c-41e4-8da9-f649fc651ae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-e833eee5-b19c-41e4-8da9-f649fc651ae2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:11.598442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:11.599352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:11.600238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 241/484] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:11.606314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833eee5-b19c-41e4-8da9-f649fc651ae2 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:11.623340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9f47c0fa-71c2-416d-ad83-ee987dd6a3de None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:11.624873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9f47c0fa-71c2-416d-ad83-ee987dd6a3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:11.713307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833eee5-b19c-41e4-8da9-f649fc651ae2 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:11.749107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eadafb-e29b-417d-a7ce-f3f6a3b11440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:11.749783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eadafb-e29b-417d-a7ce-f3f6a3b11440 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:11.750599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eadafb-e29b-417d-a7ce-f3f6a3b11440 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:11.750887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eadafb-e29b-417d-a7ce-f3f6a3b11440 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:11.752023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eadafb-e29b-417d-a7ce-f3f6a3b11440 None admin] RBAC: Adding query filter params (name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:11.752348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eadafb-e29b-417d-a7ce-f3f6a3b11440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:11.753739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eadafb-e29b-417d-a7ce-f3f6a3b11440 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:11.762945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 242/485] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:11.769174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eadafb-e29b-417d-a7ce-f3f6a3b11440 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:11.787397 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:11.796129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 244/486] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:11.833897 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47c0fa-71c2-416d-ad83-ee987dd6a3de None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:11.978665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:11.980200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eadafb-e29b-417d-a7ce-f3f6a3b11440 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:11.986109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47c0fa-71c2-416d-ad83-ee987dd6a3de None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:12.022720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab157a7e-eb1d-4290-9261-b90586657ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:12.023522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab157a7e-eb1d-4290-9261-b90586657ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.023809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab157a7e-eb1d-4290-9261-b90586657ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.024087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab157a7e-eb1d-4290-9261-b90586657ec5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.035048 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab157a7e-eb1d-4290-9261-b90586657ec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:12.041632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8252c-6b41-4ed2-8988-5a0bdc6ff6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:12.042480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8252c-6b41-4ed2-8988-5a0bdc6ff6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.042764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8252c-6b41-4ed2-8988-5a0bdc6ff6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-ab157a7e-eb1d-4290-9261-b90586657ec5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:12.043007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:12.043861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:12.044669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:12.045300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8252c-6b41-4ed2-8988-5a0bdc6ff6f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.045374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 245/487] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:12.050730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c29eb-a4a9-4017-b517-88109a728a11 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.051036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c29eb-a4a9-4017-b517-88109a728a11 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.051322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c29eb-a4a9-4017-b517-88109a728a11 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.053186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 246/488] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:12.054617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c29eb-a4a9-4017-b517-88109a728a11 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:12.058905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8252c-6b41-4ed2-8988-5a0bdc6ff6f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:12.060361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8252c-6b41-4ed2-8988-5a0bdc6ff6f7 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-24f8252c-6b41-4ed2-8988-5a0bdc6ff6f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:12.079347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:12.081420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:12.083571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:12.083571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:12.083571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:12.083571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:12.083571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 243/489] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:12.083571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81692211-c50c-4c31-aa91-4b7778e44bad None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.083571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81692211-c50c-4c31-aa91-4b7778e44bad None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.083571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81692211-c50c-4c31-aa91-4b7778e44bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.106511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-81692211-c50c-4c31-aa91-4b7778e44bad None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:12.107716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-81692211-c50c-4c31-aa91-4b7778e44bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:12.192380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:12.195748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c29eb-a4a9-4017-b517-88109a728a11 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:12.212951 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 244/490] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:12.214107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81692211-c50c-4c31-aa91-4b7778e44bad None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:12.242453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44cc68-b9fe-439d-b5bd-3120d0398fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:12.243132 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44cc68-b9fe-439d-b5bd-3120d0398fbb None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.243696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44cc68-b9fe-439d-b5bd-3120d0398fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.243994 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44cc68-b9fe-439d-b5bd-3120d0398fbb None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.244753 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44cc68-b9fe-439d-b5bd-3120d0398fbb None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:12.245063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44cc68-b9fe-439d-b5bd-3120d0398fbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:12.247033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44cc68-b9fe-439d-b5bd-3120d0398fbb None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:12.258851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 247/491] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:12.262850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44cc68-b9fe-439d-b5bd-3120d0398fbb None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:12.277292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:12.345918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81692211-c50c-4c31-aa91-4b7778e44bad None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:12.388498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44cc68-b9fe-439d-b5bd-3120d0398fbb None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:12.389495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fb649-7950-443b-9af7-d2b8b707dda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:12.390449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fb649-7950-443b-9af7-d2b8b707dda0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.391707 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fb649-7950-443b-9af7-d2b8b707dda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.392006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fb649-7950-443b-9af7-d2b8b707dda0 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.392752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fb649-7950-443b-9af7-d2b8b707dda0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:12.393064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fb649-7950-443b-9af7-d2b8b707dda0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:12.394489 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fb649-7950-443b-9af7-d2b8b707dda0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:12.414351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 245/492] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:12.416537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fb649-7950-443b-9af7-d2b8b707dda0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:12.430685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87c03e-c1d8-441f-ad3a-c3c814faf9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:12.431486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c03e-c1d8-441f-ad3a-c3c814faf9dc None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.431769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c03e-c1d8-441f-ad3a-c3c814faf9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.432041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c03e-c1d8-441f-ad3a-c3c814faf9dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.444309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87c03e-c1d8-441f-ad3a-c3c814faf9dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-0c87c03e-c1d8-441f-ad3a-c3c814faf9dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:12.452291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:12.453355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:12.454159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:12.459146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 248/493] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:12.460196 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87c03e-c1d8-441f-ad3a-c3c814faf9dc None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:12.503285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:12.535219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:12.563601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fb649-7950-443b-9af7-d2b8b707dda0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:12.573360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87c03e-c1d8-441f-ad3a-c3c814faf9dc None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:12.610516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a6a0c-0040-4c6f-a18a-0ebd96cb0eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:12.611777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6a0c-0040-4c6f-a18a-0ebd96cb0eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.612266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6a0c-0040-4c6f-a18a-0ebd96cb0eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.612793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6a0c-0040-4c6f-a18a-0ebd96cb0eb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.613315 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748dab3-8c26-4783-b5cc-aa18ca5a7ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:12.614345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748dab3-8c26-4783-b5cc-aa18ca5a7ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.614632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748dab3-8c26-4783-b5cc-aa18ca5a7ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.614905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748dab3-8c26-4783-b5cc-aa18ca5a7ce2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.623843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748dab3-8c26-4783-b5cc-aa18ca5a7ce2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:12.625445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748dab3-8c26-4783-b5cc-aa18ca5a7ce2 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:12.630325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a6a0c-0040-4c6f-a18a-0ebd96cb0eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:12.632167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a6a0c-0040-4c6f-a18a-0ebd96cb0eb6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-8748dab3-8c26-4783-b5cc-aa18ca5a7ce2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:12.643441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:12.644383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:12.645301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:12.645301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:12.645301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:12.645301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:12.645301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:12.645301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:12.645301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:12.645301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:12.645301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 12:46:12.645301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:12.645301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 249/494] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:12.645849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748dab3-8c26-4783-b5cc-aa18ca5a7ce2 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-513a6a0c-0040-4c6f-a18a-0ebd96cb0eb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:12.658484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:12.659418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:12.660372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:12.660372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:12.660372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:12.660372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:12.660372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 246/495] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:12.661202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01962325-e2f8-471b-9932-39a0d070c7bd None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.661493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01962325-e2f8-471b-9932-39a0d070c7bd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.661771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01962325-e2f8-471b-9932-39a0d070c7bd None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.663194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 247/496] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:12.667168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01962325-e2f8-471b-9932-39a0d070c7bd None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:12.770406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748dab3-8c26-4783-b5cc-aa18ca5a7ce2 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:12.775130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01962325-e2f8-471b-9932-39a0d070c7bd None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:12.784621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:12.800152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:12.809121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19276e-2d4f-4438-93a3-425da67ae10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:12.809789 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19276e-2d4f-4438-93a3-425da67ae10b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.810091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19276e-2d4f-4438-93a3-425da67ae10b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.810376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19276e-2d4f-4438-93a3-425da67ae10b None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.811118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19276e-2d4f-4438-93a3-425da67ae10b None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:12.811422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19276e-2d4f-4438-93a3-425da67ae10b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:12.812389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b612a-6846-4e78-831f-7b2d0fd45111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:12.812875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19276e-2d4f-4438-93a3-425da67ae10b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:12.813092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b612a-6846-4e78-831f-7b2d0fd45111 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.813379 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b612a-6846-4e78-831f-7b2d0fd45111 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.813657 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b612a-6846-4e78-831f-7b2d0fd45111 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.814531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b612a-6846-4e78-831f-7b2d0fd45111 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:12.814841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b612a-6846-4e78-831f-7b2d0fd45111 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:12.816266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b612a-6846-4e78-831f-7b2d0fd45111 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:12.820903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 250/497] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:12.821678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19276e-2d4f-4438-93a3-425da67ae10b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:12.854189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 248/498] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 12:46:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:12.855484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58d0f9-765b-44f6-b258-dc66021cc0a7 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.855775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58d0f9-765b-44f6-b258-dc66021cc0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.856063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58d0f9-765b-44f6-b258-dc66021cc0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.872166 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-ac58d0f9-765b-44f6-b258-dc66021cc0a7 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:12.873340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ac58d0f9-765b-44f6-b258-dc66021cc0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:12.926743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19276e-2d4f-4438-93a3-425da67ae10b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:12.938148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 249/499] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:12.939473 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e644c1-f8cb-468f-b73b-9ef988bcb3b9 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.939764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e644c1-f8cb-468f-b73b-9ef988bcb3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.940050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e644c1-f8cb-468f-b73b-9ef988bcb3b9 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.941299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 250/500] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:12.941986 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e644c1-f8cb-468f-b73b-9ef988bcb3b9 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:12.965635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259927-245a-4a07-bcf0-ddb4e0dee5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:12.966327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259927-245a-4a07-bcf0-ddb4e0dee5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:12.966621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259927-245a-4a07-bcf0-ddb4e0dee5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:12.966898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259927-245a-4a07-bcf0-ddb4e0dee5a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:12.967614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6259927-245a-4a07-bcf0-ddb4e0dee5a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:12.967918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6259927-245a-4a07-bcf0-ddb4e0dee5a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:12.969316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6259927-245a-4a07-bcf0-ddb4e0dee5a0 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:12.980733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 251/501] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:12.981492 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259927-245a-4a07-bcf0-ddb4e0dee5a0 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:13.029730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:13.046662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:13.053372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e644c1-f8cb-468f-b73b-9ef988bcb3b9 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:13.079853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259927-245a-4a07-bcf0-ddb4e0dee5a0 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:13.085887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8be10d-1468-42a9-821c-9ac8e0573101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:13.086712 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8be10d-1468-42a9-821c-9ac8e0573101 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.087003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8be10d-1468-42a9-821c-9ac8e0573101 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.087280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8be10d-1468-42a9-821c-9ac8e0573101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.097721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8be10d-1468-42a9-821c-9ac8e0573101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:13.099203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8be10d-1468-42a9-821c-9ac8e0573101 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:13.113159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6edac1-7f91-4211-b76a-ff518b4765ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:13.113798 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6edac1-7f91-4211-b76a-ff518b4765ad None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.114081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6edac1-7f91-4211-b76a-ff518b4765ad None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.114389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6edac1-7f91-4211-b76a-ff518b4765ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-cf8be10d-1468-42a9-821c-9ac8e0573101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:13.116403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:13.117224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:13.122919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:13.122919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:13.122919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:13.122919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:13.122919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 251/502] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:13.122919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414414d-daf8-4ab7-aa5e-2218b4ae3925 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.122919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414414d-daf8-4ab7-aa5e-2218b4ae3925 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.122919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414414d-daf8-4ab7-aa5e-2218b4ae3925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.136276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-d414414d-daf8-4ab7-aa5e-2218b4ae3925 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:13.137478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d414414d-daf8-4ab7-aa5e-2218b4ae3925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:13.182253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6edac1-7f91-4211-b76a-ff518b4765ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:13.183707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6edac1-7f91-4211-b76a-ff518b4765ad None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:13.207521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 252/503] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:13.208353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414414d-daf8-4ab7-aa5e-2218b4ae3925 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:13.251090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:13.284027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:13.300609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 252/504] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:13.301428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6edac1-7f91-4211-b76a-ff518b4765ad None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:13.310905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414414d-daf8-4ab7-aa5e-2218b4ae3925 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:13.344191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5499e5-2c89-4a40-aa18-1fbaeca1fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:13.344985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5499e5-2c89-4a40-aa18-1fbaeca1fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.345267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5499e5-2c89-4a40-aa18-1fbaeca1fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.345539 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5499e5-2c89-4a40-aa18-1fbaeca1fbf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.355290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5499e5-2c89-4a40-aa18-1fbaeca1fbf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:13.356771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5499e5-2c89-4a40-aa18-1fbaeca1fbf2 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-3c5499e5-2c89-4a40-aa18-1fbaeca1fbf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:13.374650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:13.375620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:13.376555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:13.376555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:13.376555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:13.376555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:13.376555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 253/505] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:13.376817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5499e5-2c89-4a40-aa18-1fbaeca1fbf2 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:13.405279 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6edac1-7f91-4211-b76a-ff518b4765ad None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:13.440121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b0ea4-0fb4-4a1a-852c-1a192acc2ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:13.440841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b0ea4-0fb4-4a1a-852c-1a192acc2ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.441130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b0ea4-0fb4-4a1a-852c-1a192acc2ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.441404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b0ea4-0fb4-4a1a-852c-1a192acc2ad8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.448344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b0ea4-0fb4-4a1a-852c-1a192acc2ad8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:13.449868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b0ea4-0fb4-4a1a-852c-1a192acc2ad8 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-ed8b0ea4-0fb4-4a1a-852c-1a192acc2ad8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:13.464103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:13.465121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 253/506] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 12:46:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:13.466017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b0ea4-0fb4-4a1a-852c-1a192acc2ad8 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:13.477401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:13.483516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5499e5-2c89-4a40-aa18-1fbaeca1fbf2 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:13.513736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:13.519994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b249fef-3872-4c1b-82d0-0eb6e65293a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:13.520661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b249fef-3872-4c1b-82d0-0eb6e65293a4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.520941 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b249fef-3872-4c1b-82d0-0eb6e65293a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.521215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b249fef-3872-4c1b-82d0-0eb6e65293a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.521937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b249fef-3872-4c1b-82d0-0eb6e65293a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:13.522248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b249fef-3872-4c1b-82d0-0eb6e65293a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:13.523644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b249fef-3872-4c1b-82d0-0eb6e65293a4 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:13.535194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 254/507] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:13.536217 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b249fef-3872-4c1b-82d0-0eb6e65293a4 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:13.569259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b0ea4-0fb4-4a1a-852c-1a192acc2ad8 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:13.601573 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2bc8a-cf32-43ff-b050-9a11353a6734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:13.602389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2bc8a-cf32-43ff-b050-9a11353a6734 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.602678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2bc8a-cf32-43ff-b050-9a11353a6734 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.602958 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2bc8a-cf32-43ff-b050-9a11353a6734 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.613316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2bc8a-cf32-43ff-b050-9a11353a6734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-2ff2bc8a-cf32-43ff-b050-9a11353a6734 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:13.621029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:13.621957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:13.622787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 254/508] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:13.623639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2bc8a-cf32-43ff-b050-9a11353a6734 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:13.635978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b249fef-3872-4c1b-82d0-0eb6e65293a4 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:13.668896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab86919-c910-4025-b210-0bcebc534ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:13.669738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab86919-c910-4025-b210-0bcebc534ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.670027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab86919-c910-4025-b210-0bcebc534ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.670323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab86919-c910-4025-b210-0bcebc534ac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.681041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab86919-c910-4025-b210-0bcebc534ac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:13.682507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab86919-c910-4025-b210-0bcebc534ac0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:13.691019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-8ab86919-c910-4025-b210-0bcebc534ac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:13.699613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:13.700660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:13.701575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:13.701575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:13.701575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:13.701575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:13.701575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 255/509] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:13.701575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab86919-c910-4025-b210-0bcebc534ac0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:13.724506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2bc8a-cf32-43ff-b050-9a11353a6734 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:13.744901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:13.759075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fabbd-dd0e-4d0d-b78f-731ec74d2b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:13.759735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fabbd-dd0e-4d0d-b78f-731ec74d2b8e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.760028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fabbd-dd0e-4d0d-b78f-731ec74d2b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.760313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fabbd-dd0e-4d0d-b78f-731ec74d2b8e None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.761282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fabbd-dd0e-4d0d-b78f-731ec74d2b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:13.761603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fabbd-dd0e-4d0d-b78f-731ec74d2b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:13.763039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fabbd-dd0e-4d0d-b78f-731ec74d2b8e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:13.774438 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 255/510] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:13.775218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fabbd-dd0e-4d0d-b78f-731ec74d2b8e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:13.805171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab86919-c910-4025-b210-0bcebc534ac0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:13.837453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e4b9b-7b4c-40cf-9cb1-c9013d85355e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:13.838125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e4b9b-7b4c-40cf-9cb1-c9013d85355e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.838457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e4b9b-7b4c-40cf-9cb1-c9013d85355e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.838741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e4b9b-7b4c-40cf-9cb1-c9013d85355e None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.839567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e4b9b-7b4c-40cf-9cb1-c9013d85355e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:13.839884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e4b9b-7b4c-40cf-9cb1-c9013d85355e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:13.841300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e4b9b-7b4c-40cf-9cb1-c9013d85355e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:13.858370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 256/511] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 12:46:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:13.859156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e4b9b-7b4c-40cf-9cb1-c9013d85355e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:13.871514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fabbd-dd0e-4d0d-b78f-731ec74d2b8e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:13.898829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:13.904941 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4265dac-ba56-4995-9544-ff2647ec682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:13.905715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4265dac-ba56-4995-9544-ff2647ec682e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.906001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4265dac-ba56-4995-9544-ff2647ec682e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.906827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4265dac-ba56-4995-9544-ff2647ec682e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.913752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4265dac-ba56-4995-9544-ff2647ec682e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:13.915228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4265dac-ba56-4995-9544-ff2647ec682e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-b4265dac-ba56-4995-9544-ff2647ec682e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:13.930001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:13.930966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 256/512] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:13.932069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4265dac-ba56-4995-9544-ff2647ec682e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:13.956613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:13.959721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e4b9b-7b4c-40cf-9cb1-c9013d85355e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:13.993561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9bba6-647a-4574-98ac-c09eddae6616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:13.994223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9bba6-647a-4574-98ac-c09eddae6616 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:13.994531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9bba6-647a-4574-98ac-c09eddae6616 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:13.994802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9bba6-647a-4574-98ac-c09eddae6616 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:13.995537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9bba6-647a-4574-98ac-c09eddae6616 None admin] RBAC: Adding query filter params (name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:13.995842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9bba6-647a-4574-98ac-c09eddae6616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:13.997200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9bba6-647a-4574-98ac-c09eddae6616 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:14.005606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 257/513] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:14.006389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9bba6-647a-4574-98ac-c09eddae6616 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:14.032540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4265dac-ba56-4995-9544-ff2647ec682e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:14.065807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233bf0a-c9ee-4480-b8eb-054fb73da824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:14.066503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233bf0a-c9ee-4480-b8eb-054fb73da824 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:14.066788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233bf0a-c9ee-4480-b8eb-054fb73da824 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:14.067065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233bf0a-c9ee-4480-b8eb-054fb73da824 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:14.067782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233bf0a-c9ee-4480-b8eb-054fb73da824 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:14.068088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233bf0a-c9ee-4480-b8eb-054fb73da824 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:14.069466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233bf0a-c9ee-4480-b8eb-054fb73da824 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:14.081147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 257/514] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:14.081743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233bf0a-c9ee-4480-b8eb-054fb73da824 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:14.104151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9bba6-647a-4574-98ac-c09eddae6616 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:14.129543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:14.137193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61c0fd-3bd2-43fd-829f-1d42c130bb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:14.137877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61c0fd-3bd2-43fd-829f-1d42c130bb07 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:14.138159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61c0fd-3bd2-43fd-829f-1d42c130bb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:14.138470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61c0fd-3bd2-43fd-829f-1d42c130bb07 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:14.139168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61c0fd-3bd2-43fd-829f-1d42c130bb07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:14.141123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61c0fd-3bd2-43fd-829f-1d42c130bb07 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:14.161484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 258/515] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:46:14 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:46:14.162648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61c0fd-3bd2-43fd-829f-1d42c130bb07 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:14.185731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:14.193861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233bf0a-c9ee-4480-b8eb-054fb73da824 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:14.242155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39a605-5ec8-4642-bfb4-aebbb79a92b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:14.242770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39a605-5ec8-4642-bfb4-aebbb79a92b4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:14.243056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39a605-5ec8-4642-bfb4-aebbb79a92b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:14.243335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39a605-5ec8-4642-bfb4-aebbb79a92b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:14.300078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61c0fd-3bd2-43fd-829f-1d42c130bb07 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:14.325662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39a605-5ec8-4642-bfb4-aebbb79a92b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:14.327185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39a605-5ec8-4642-bfb4-aebbb79a92b4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:14.340195 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7262829-dcd5-496a-b6ec-53213271a736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:14.341045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262829-dcd5-496a-b6ec-53213271a736 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:14.341332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262829-dcd5-496a-b6ec-53213271a736 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:14.341609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262829-dcd5-496a-b6ec-53213271a736 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:14.342486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7262829-dcd5-496a-b6ec-53213271a736 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:14.342800 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7262829-dcd5-496a-b6ec-53213271a736 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:14.344874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7262829-dcd5-496a-b6ec-53213271a736 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:14.365128 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 259/516] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 12:46:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:14.366056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7262829-dcd5-496a-b6ec-53213271a736 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:14.402110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:14.452303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 258/517] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:14.453238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39a605-5ec8-4642-bfb4-aebbb79a92b4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:14.457686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:14.482496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7262829-dcd5-496a-b6ec-53213271a736 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:14.521650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc07ce-c32d-4e4d-a837-2e861cfec5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:14.522958 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc07ce-c32d-4e4d-a837-2e861cfec5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:14.523483 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc07ce-c32d-4e4d-a837-2e861cfec5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:14.524388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc07ce-c32d-4e4d-a837-2e861cfec5d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:14.537368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc07ce-c32d-4e4d-a837-2e861cfec5d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:14.539268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc07ce-c32d-4e4d-a837-2e861cfec5d9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-d3bc07ce-c32d-4e4d-a837-2e861cfec5d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:14.560841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:14.561727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:14.563145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:14.563145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:14.563145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:14.563145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:14.564477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 260/518] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:14.565137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc07ce-c32d-4e4d-a837-2e861cfec5d9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:14.571936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39a605-5ec8-4642-bfb4-aebbb79a92b4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:14.620188 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6c920-23c9-4c80-a709-7716e55e9fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:14.620974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6c920-23c9-4c80-a709-7716e55e9fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:14.621259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6c920-23c9-4c80-a709-7716e55e9fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:14.621539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6c920-23c9-4c80-a709-7716e55e9fa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:14.628771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6c920-23c9-4c80-a709-7716e55e9fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:14.630237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6c920-23c9-4c80-a709-7716e55e9fa5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-13d6c920-23c9-4c80-a709-7716e55e9fa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:14.645205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:14.646256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:14.647173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:14.647173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:14.647173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:14.647173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:14.647173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:14.647173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:14.647173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:14.647173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:14.647173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 259/519] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:14.647173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6c920-23c9-4c80-a709-7716e55e9fa5 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:14.662378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:14.691288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc07ce-c32d-4e4d-a837-2e861cfec5d9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:14.720580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:14.728848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e172c-30db-4ebe-afbd-58e4ced107bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:14.729506 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e172c-30db-4ebe-afbd-58e4ced107bb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:14.729792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e172c-30db-4ebe-afbd-58e4ced107bb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:14.730070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e172c-30db-4ebe-afbd-58e4ced107bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:14.760064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6c920-23c9-4c80-a709-7716e55e9fa5 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:14.795232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f104001-5cb3-4549-9578-0ecc64c1612a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:14.796252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f104001-5cb3-4549-9578-0ecc64c1612a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:14.796536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f104001-5cb3-4549-9578-0ecc64c1612a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:14.796840 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f104001-5cb3-4549-9578-0ecc64c1612a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:14.798371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7e172c-30db-4ebe-afbd-58e4ced107bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:14.799821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7e172c-30db-4ebe-afbd-58e4ced107bb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:14.808511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f104001-5cb3-4549-9578-0ecc64c1612a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:14.809966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f104001-5cb3-4549-9578-0ecc64c1612a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-0f104001-5cb3-4549-9578-0ecc64c1612a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:14.827466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:14.828383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:14.829224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:14.829224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:14.829224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:14.829224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:14.829224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 260/520] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:14.829483 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f104001-5cb3-4549-9578-0ecc64c1612a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:14.896564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:14.907787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 261/521] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:14.908699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e172c-30db-4ebe-afbd-58e4ced107bb None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:14.938426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:14.945220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f104001-5cb3-4549-9578-0ecc64c1612a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:14.986387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3046d4-e1e9-4e33-bf25-06ae765a32b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:14.987224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3046d4-e1e9-4e33-bf25-06ae765a32b3 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:14.987507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3046d4-e1e9-4e33-bf25-06ae765a32b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:14.987771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3046d4-e1e9-4e33-bf25-06ae765a32b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:14.988497 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3046d4-e1e9-4e33-bf25-06ae765a32b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:14.988806 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3046d4-e1e9-4e33-bf25-06ae765a32b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:14.990194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3046d4-e1e9-4e33-bf25-06ae765a32b3 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:15.007299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 261/522] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:15.007961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3046d4-e1e9-4e33-bf25-06ae765a32b3 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:15.018098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e172c-30db-4ebe-afbd-58e4ced107bb None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:15.057003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd447a-01ae-4c70-ab57-f614b1126c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:15.057595 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd447a-01ae-4c70-ab57-f614b1126c28 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:15.057878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd447a-01ae-4c70-ab57-f614b1126c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:15.058174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd447a-01ae-4c70-ab57-f614b1126c28 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:15.058971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd447a-01ae-4c70-ab57-f614b1126c28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:15.059281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd447a-01ae-4c70-ab57-f614b1126c28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:15.060711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd447a-01ae-4c70-ab57-f614b1126c28 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:15.080408 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 262/523] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:15.081482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd447a-01ae-4c70-ab57-f614b1126c28 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:15.114955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3046d4-e1e9-4e33-bf25-06ae765a32b3 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:15.137942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:15.150700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7ff55-fa24-44a9-9506-20fde0348d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:15.151366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7ff55-fa24-44a9-9506-20fde0348d21 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:15.151651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7ff55-fa24-44a9-9506-20fde0348d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:15.151931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7ff55-fa24-44a9-9506-20fde0348d21 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:15.152650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7ff55-fa24-44a9-9506-20fde0348d21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:15.152959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7ff55-fa24-44a9-9506-20fde0348d21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:15.154350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7ff55-fa24-44a9-9506-20fde0348d21 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:15.170429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 262/524] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:15.170429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7ff55-fa24-44a9-9506-20fde0348d21 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:15.176888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:15.190490 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd447a-01ae-4c70-ab57-f614b1126c28 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:15.225484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90c815-20a7-46d4-bd2c-aaeacfb9e6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:15.226307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c815-20a7-46d4-bd2c-aaeacfb9e6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:15.226592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c815-20a7-46d4-bd2c-aaeacfb9e6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:15.226864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c815-20a7-46d4-bd2c-aaeacfb9e6a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:15.236749 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90c815-20a7-46d4-bd2c-aaeacfb9e6a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:15.238177 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90c815-20a7-46d4-bd2c-aaeacfb9e6a2 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-2b90c815-20a7-46d4-bd2c-aaeacfb9e6a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:15.256467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:15.257604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:15.258536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:15.258536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:15.258536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:15.258536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:15.258536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 263/525] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:15.258873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90c815-20a7-46d4-bd2c-aaeacfb9e6a2 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:15.276179 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7ff55-fa24-44a9-9506-20fde0348d21 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:15.315409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef851876-7481-441d-a460-edaa7568a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:15.316241 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef851876-7481-441d-a460-edaa7568a62d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:15.316526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef851876-7481-441d-a460-edaa7568a62d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:15.317358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef851876-7481-441d-a460-edaa7568a62d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:15.326331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef851876-7481-441d-a460-edaa7568a62d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:15.327847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef851876-7481-441d-a460-edaa7568a62d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-ef851876-7481-441d-a460-edaa7568a62d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:15.343046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:15.344511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 263/526] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:15.345485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef851876-7481-441d-a460-edaa7568a62d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:15.374327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90c815-20a7-46d4-bd2c-aaeacfb9e6a2 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:15.393929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:15.418511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06713d56-4f09-41a4-9fdb-dfe19c8c8156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:15.419181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06713d56-4f09-41a4-9fdb-dfe19c8c8156 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:15.419466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06713d56-4f09-41a4-9fdb-dfe19c8c8156 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:15.419764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06713d56-4f09-41a4-9fdb-dfe19c8c8156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:15.420582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06713d56-4f09-41a4-9fdb-dfe19c8c8156 None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=e47f1d65687e4cfcbce764076141702c) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:15.420895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06713d56-4f09-41a4-9fdb-dfe19c8c8156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:15.422457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06713d56-4f09-41a4-9fdb-dfe19c8c8156 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:15.438320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 264/527] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:15 2022] GET /identity/v3/role_assignments?user.id=e47f1d65687e4cfcbce764076141702c&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:15.438320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06713d56-4f09-41a4-9fdb-dfe19c8c8156 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:15.440280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:15.470769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef851876-7481-441d-a460-edaa7568a62d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:15.512851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625c3a-10b9-4acd-b8c7-23d7ae5cef5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:15.513699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625c3a-10b9-4acd-b8c7-23d7ae5cef5e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:15.513988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625c3a-10b9-4acd-b8c7-23d7ae5cef5e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:15.514268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625c3a-10b9-4acd-b8c7-23d7ae5cef5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:15.522907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1625c3a-10b9-4acd-b8c7-23d7ae5cef5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:15.524491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1625c3a-10b9-4acd-b8c7-23d7ae5cef5e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-c1625c3a-10b9-4acd-b8c7-23d7ae5cef5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:15.541470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:15.542465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:15.543323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:15.543323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:15.543323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:15.543323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:15.543323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:15.543323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:15.543323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:15.543323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:15.543323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 264/528] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:15.543795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625c3a-10b9-4acd-b8c7-23d7ae5cef5e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:15.586767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06713d56-4f09-41a4-9fdb-dfe19c8c8156 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:15.636873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce4aab-2960-4c40-9643-29b47ced82c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:15.637861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce4aab-2960-4c40-9643-29b47ced82c5 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:15.638350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce4aab-2960-4c40-9643-29b47ced82c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:15.638829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce4aab-2960-4c40-9643-29b47ced82c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:15.639751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce4aab-2960-4c40-9643-29b47ced82c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:15.640063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce4aab-2960-4c40-9643-29b47ced82c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:15.641977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce4aab-2960-4c40-9643-29b47ced82c5 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:15.654396 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 265/529] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:15.655288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce4aab-2960-4c40-9643-29b47ced82c5 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:15.668580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625c3a-10b9-4acd-b8c7-23d7ae5cef5e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:15.694164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:15.717927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c18b6-6ad3-4dde-8e83-e3f527b4e605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:15.719281 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c18b6-6ad3-4dde-8e83-e3f527b4e605 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:15.719806 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c18b6-6ad3-4dde-8e83-e3f527b4e605 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:15.720540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c18b6-6ad3-4dde-8e83-e3f527b4e605 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:15.722260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c18b6-6ad3-4dde-8e83-e3f527b4e605 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:15.722843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c18b6-6ad3-4dde-8e83-e3f527b4e605 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:15.724413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:15.726468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c18b6-6ad3-4dde-8e83-e3f527b4e605 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:15.766574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce4aab-2960-4c40-9643-29b47ced82c5 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:15.775224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 265/530] 10.222.0.45 () {56 vars in 1139 bytes} [Tue May 24 12:46:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:15.776422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c18b6-6ad3-4dde-8e83-e3f527b4e605 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:15.812963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb522a6-0155-4428-894b-ab40198919c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:15.814423 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb522a6-0155-4428-894b-ab40198919c6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:15.815182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb522a6-0155-4428-894b-ab40198919c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:15.815615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb522a6-0155-4428-894b-ab40198919c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:15.816653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb522a6-0155-4428-894b-ab40198919c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:15.817116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb522a6-0155-4428-894b-ab40198919c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:15.819941 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb522a6-0155-4428-894b-ab40198919c6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:15.836633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 266/531] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:15.837747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb522a6-0155-4428-894b-ab40198919c6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:15.888361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c18b6-6ad3-4dde-8e83-e3f527b4e605 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:15.925568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f679c7-9cb2-4a5a-bdaa-57b48453ec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:15.926552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f679c7-9cb2-4a5a-bdaa-57b48453ec53 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:15.926954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f679c7-9cb2-4a5a-bdaa-57b48453ec53 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:15.927331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f679c7-9cb2-4a5a-bdaa-57b48453ec53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:15.934594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f679c7-9cb2-4a5a-bdaa-57b48453ec53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:15.936199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f679c7-9cb2-4a5a-bdaa-57b48453ec53 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-93f679c7-9cb2-4a5a-bdaa-57b48453ec53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:15.951998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:15.952899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:15.953802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:15.953802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:15.953802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:15.953802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:15.953802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:15.953802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:15.953802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:15.953802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:15.953802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 266/532] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:15.954760 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f679c7-9cb2-4a5a-bdaa-57b48453ec53 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:15.965235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb522a6-0155-4428-894b-ab40198919c6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:15.977372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:15.996855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:16.010719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c06ea-8779-4b1d-99aa-4d85ff3651aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:16.011643 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c06ea-8779-4b1d-99aa-4d85ff3651aa None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:16.011930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c06ea-8779-4b1d-99aa-4d85ff3651aa None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:16.012207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c06ea-8779-4b1d-99aa-4d85ff3651aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:16.076043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f679c7-9cb2-4a5a-bdaa-57b48453ec53 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:16.098141 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c06ea-8779-4b1d-99aa-4d85ff3651aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:16.099872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c06ea-8779-4b1d-99aa-4d85ff3651aa None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:16.113134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a3a20-e620-4344-b495-d5fbc1c55ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:16.114072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a3a20-e620-4344-b495-d5fbc1c55ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:16.114595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a3a20-e620-4344-b495-d5fbc1c55ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:16.115112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a3a20-e620-4344-b495-d5fbc1c55ab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:16.116142 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a3a20-e620-4344-b495-d5fbc1c55ab4 None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=05b2edb6fcb7496bb55650080a9da172) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:16.116667 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a3a20-e620-4344-b495-d5fbc1c55ab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:16.118326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a3a20-e620-4344-b495-d5fbc1c55ab4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:16.124845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 267/533] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:15 2022] GET /identity/v3/role_assignments?user.id=05b2edb6fcb7496bb55650080a9da172&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:16.126446 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a3a20-e620-4344-b495-d5fbc1c55ab4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:16.222452 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:16.225098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 267/534] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:16.235490 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c06ea-8779-4b1d-99aa-4d85ff3651aa None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:16.250661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a3a20-e620-4344-b495-d5fbc1c55ab4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:16.287877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:16.293185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5d62b-07d8-447b-8db2-4a30025c0461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:16.294264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5d62b-07d8-447b-8db2-4a30025c0461 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:16.294824 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5d62b-07d8-447b-8db2-4a30025c0461 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:16.295542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5d62b-07d8-447b-8db2-4a30025c0461 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:16.296568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5d62b-07d8-447b-8db2-4a30025c0461 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:16.297147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5d62b-07d8-447b-8db2-4a30025c0461 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:16.298888 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5d62b-07d8-447b-8db2-4a30025c0461 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:16.319931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 268/535] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:16.325303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5d62b-07d8-447b-8db2-4a30025c0461 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:16.456328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c06ea-8779-4b1d-99aa-4d85ff3651aa None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:16.475177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5d62b-07d8-447b-8db2-4a30025c0461 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:16.502333 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cb528-ae7b-4b57-892c-917560fb54ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:16.503393 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cb528-ae7b-4b57-892c-917560fb54ff None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:16.504118 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cb528-ae7b-4b57-892c-917560fb54ff None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:16.504402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cb528-ae7b-4b57-892c-917560fb54ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:16.511770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76495e-c7cd-4a4a-b7fc-0f7014a7ef87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:16.512562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76495e-c7cd-4a4a-b7fc-0f7014a7ef87 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:16.512855 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76495e-c7cd-4a4a-b7fc-0f7014a7ef87 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:16.513133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76495e-c7cd-4a4a-b7fc-0f7014a7ef87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:16.513973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76495e-c7cd-4a4a-b7fc-0f7014a7ef87 None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=8c9f08515d514be8af1176717ec9e6fd) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:16.514419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76495e-c7cd-4a4a-b7fc-0f7014a7ef87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:16.516431 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76495e-c7cd-4a4a-b7fc-0f7014a7ef87 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:16.516808 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8cb528-ae7b-4b57-892c-917560fb54ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:16.518531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8cb528-ae7b-4b57-892c-917560fb54ff None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:16.522697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 269/536] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:16 2022] GET /identity/v3/role_assignments?user.id=8c9f08515d514be8af1176717ec9e6fd&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:16.523940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd22ea-2658-4ebb-83ba-9ae07ab21f02 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:16.524228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd22ea-2658-4ebb-83ba-9ae07ab21f02 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:16.524508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd22ea-2658-4ebb-83ba-9ae07ab21f02 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:16.525724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 270/537] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:16.536461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41469b2-1850-45de-a94d-c790b565be29 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:16.536777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41469b2-1850-45de-a94d-c790b565be29 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:16.537023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41469b2-1850-45de-a94d-c790b565be29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-ee8cb528-ae7b-4b57-892c-917560fb54ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:16.542870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:16.543769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:16.544632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:16.544632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:16.544632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:16.544632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:16.544632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 268/538] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:16.551371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cb528-ae7b-4b57-892c-917560fb54ff None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:16.579739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f41469b2-1850-45de-a94d-c790b565be29 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:16.580970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f41469b2-1850-45de-a94d-c790b565be29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:16.583435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:16.694883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:16.714839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 271/539] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:16.728004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cb528-ae7b-4b57-892c-917560fb54ff None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:16.739031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41469b2-1850-45de-a94d-c790b565be29 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:16.809180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42aaa9-253c-458a-b12a-8f2ebbf50323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:16.809860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42aaa9-253c-458a-b12a-8f2ebbf50323 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:16.810144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42aaa9-253c-458a-b12a-8f2ebbf50323 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:16.810475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42aaa9-253c-458a-b12a-8f2ebbf50323 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:16.811211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42aaa9-253c-458a-b12a-8f2ebbf50323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:16.811519 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42aaa9-253c-458a-b12a-8f2ebbf50323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:16.813440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42aaa9-253c-458a-b12a-8f2ebbf50323 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:16.827130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 269/540] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:16.833153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42aaa9-253c-458a-b12a-8f2ebbf50323 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:16.929172 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41469b2-1850-45de-a94d-c790b565be29 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:16.991645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f98f08-5f13-4a0b-a243-eff26cef37e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:16.992441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f98f08-5f13-4a0b-a243-eff26cef37e7 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:16.992727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f98f08-5f13-4a0b-a243-eff26cef37e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:16.993187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f98f08-5f13-4a0b-a243-eff26cef37e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:17.010636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:17.019278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f98f08-5f13-4a0b-a243-eff26cef37e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:17.022653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42aaa9-253c-458a-b12a-8f2ebbf50323 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-65f98f08-5f13-4a0b-a243-eff26cef37e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:17.039596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:17.040555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:17.041355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 272/541] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:17.046620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f98f08-5f13-4a0b-a243-eff26cef37e7 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:17.066349 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9397a5-9b59-415b-a86d-9bf73071451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:17.068298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9397a5-9b59-415b-a86d-9bf73071451b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:17.068574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9397a5-9b59-415b-a86d-9bf73071451b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:17.068842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9397a5-9b59-415b-a86d-9bf73071451b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:17.076364 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:17.086861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9397a5-9b59-415b-a86d-9bf73071451b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:17.089692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9397a5-9b59-415b-a86d-9bf73071451b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-4f9397a5-9b59-415b-a86d-9bf73071451b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:17.126465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:17.127367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:17.128222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:17.128222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:17.128222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:17.128222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:17.128222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:17.128222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:17.128222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:17.128222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:17.130386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 270/542] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:17.146415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9397a5-9b59-415b-a86d-9bf73071451b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:17.204635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f98f08-5f13-4a0b-a243-eff26cef37e7 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:17.243086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f9e57-ff67-49b4-92e0-8482be36c7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:17.243676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f9e57-ff67-49b4-92e0-8482be36c7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:17.243959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f9e57-ff67-49b4-92e0-8482be36c7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:17.244250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f9e57-ff67-49b4-92e0-8482be36c7d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:17.244985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f9e57-ff67-49b4-92e0-8482be36c7d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:17.245291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f9e57-ff67-49b4-92e0-8482be36c7d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:17.246691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f9e57-ff67-49b4-92e0-8482be36c7d8 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:17.255467 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 273/543] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:17.260832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f9e57-ff67-49b4-92e0-8482be36c7d8 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:17.333578 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9397a5-9b59-415b-a86d-9bf73071451b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:17.358692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:17.380829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09023b-7647-4489-84d3-9a88e2d93ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:17.381520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09023b-7647-4489-84d3-9a88e2d93ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:17.381804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09023b-7647-4489-84d3-9a88e2d93ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:17.382081 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09023b-7647-4489-84d3-9a88e2d93ad6 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:17.382934 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09023b-7647-4489-84d3-9a88e2d93ad6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:17.383248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09023b-7647-4489-84d3-9a88e2d93ad6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:17.384844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09023b-7647-4489-84d3-9a88e2d93ad6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:17.398957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 271/544] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:17.404656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09023b-7647-4489-84d3-9a88e2d93ad6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:17.451762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f9e57-ff67-49b4-92e0-8482be36c7d8 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:17.520005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:17.524720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac119c-d4c6-4286-adc6-f5c3c9b5a58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:17.525528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac119c-d4c6-4286-adc6-f5c3c9b5a58c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:17.525814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac119c-d4c6-4286-adc6-f5c3c9b5a58c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:17.526086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac119c-d4c6-4286-adc6-f5c3c9b5a58c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:17.557768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac119c-d4c6-4286-adc6-f5c3c9b5a58c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:17.559282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac119c-d4c6-4286-adc6-f5c3c9b5a58c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-b9ac119c-d4c6-4286-adc6-f5c3c9b5a58c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:17.586110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:17.587056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:17.587901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:17.587901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:17.587901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:17.587901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:17.591135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09023b-7647-4489-84d3-9a88e2d93ad6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:17.595579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 274/545] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:17.611205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac119c-d4c6-4286-adc6-f5c3c9b5a58c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:17.632745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ac2c0-4f43-413e-aeeb-b32e0d4862ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:17.633413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ac2c0-4f43-413e-aeeb-b32e0d4862ac None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:17.633848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ac2c0-4f43-413e-aeeb-b32e0d4862ac None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:17.634174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ac2c0-4f43-413e-aeeb-b32e0d4862ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:17.635041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ac2c0-4f43-413e-aeeb-b32e0d4862ac None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=bb111be6bc2946d3b50b8d99a89cf8ac) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:17.635358 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ac2c0-4f43-413e-aeeb-b32e0d4862ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:17.637226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ac2c0-4f43-413e-aeeb-b32e0d4862ac None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:17.643566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 272/546] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:17 2022] GET /identity/v3/role_assignments?user.id=bb111be6bc2946d3b50b8d99a89cf8ac&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:17.784860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac119c-d4c6-4286-adc6-f5c3c9b5a58c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:17.801742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:17.831577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec041b6c-61fb-4436-bc1e-c6a766c10b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:17.832265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec041b6c-61fb-4436-bc1e-c6a766c10b57 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:17.832546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec041b6c-61fb-4436-bc1e-c6a766c10b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:17.832822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec041b6c-61fb-4436-bc1e-c6a766c10b57 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:17.833564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec041b6c-61fb-4436-bc1e-c6a766c10b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:17.833871 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec041b6c-61fb-4436-bc1e-c6a766c10b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:17.835308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec041b6c-61fb-4436-bc1e-c6a766c10b57 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:17.854664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 275/547] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:17.860817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ac2c0-4f43-413e-aeeb-b32e0d4862ac None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:17.983852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ac2c0-4f43-413e-aeeb-b32e0d4862ac None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:18.005100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:18.022475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca18737-bffb-4627-932e-0d8db33f76ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:18.023288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca18737-bffb-4627-932e-0d8db33f76ac None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:18.023571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca18737-bffb-4627-932e-0d8db33f76ac None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:18.023847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca18737-bffb-4627-932e-0d8db33f76ac None admin] PATH_INFO: `/v3/users/swift` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:18.031952 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca18737-bffb-4627-932e-0d8db33f76ac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:18.033391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca18737-bffb-4627-932e-0d8db33f76ac None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-5ca18737-bffb-4627-932e-0d8db33f76ac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:18.049604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:18.052311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:18.054737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:18.054737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:18.054737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:18.054737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:18.054737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:18.054737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:18.054737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:18.054737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:18.054737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:18.054737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:18.054737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 273/548] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:18.062351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec041b6c-61fb-4436-bc1e-c6a766c10b57 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:18.079819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa33921-adfb-4492-92c9-e7ba37c2be40 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:18.079819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa33921-adfb-4492-92c9-e7ba37c2be40 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:18.079819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa33921-adfb-4492-92c9-e7ba37c2be40 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:18.080655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 274/549] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:18.085753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd4b0-4b83-4027-a1f6-9152c5030949 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:18.086336 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd4b0-4b83-4027-a1f6-9152c5030949 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:18.086836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd4b0-4b83-4027-a1f6-9152c5030949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:18.123292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-839cd4b0-4b83-4027-a1f6-9152c5030949 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:18.124757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-839cd4b0-4b83-4027-a1f6-9152c5030949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:18.198121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec041b6c-61fb-4436-bc1e-c6a766c10b57 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:18.243592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5272648-8656-4ca9-b769-30b39a0eef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:18.244268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272648-8656-4ca9-b769-30b39a0eef76 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:18.244570 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272648-8656-4ca9-b769-30b39a0eef76 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:18.244836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272648-8656-4ca9-b769-30b39a0eef76 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:18.245687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5272648-8656-4ca9-b769-30b39a0eef76 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:18.245996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5272648-8656-4ca9-b769-30b39a0eef76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:18.247489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5272648-8656-4ca9-b769-30b39a0eef76 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:18.260021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 275/550] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:46:18.271587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 276/551] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 12:46:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:18.272920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cd4b0-4b83-4027-a1f6-9152c5030949 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:18.286005 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5272648-8656-4ca9-b769-30b39a0eef76 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:18.342621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:18.452144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cd4b0-4b83-4027-a1f6-9152c5030949 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:18.474217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5272648-8656-4ca9-b769-30b39a0eef76 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:18.476336 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:18.519530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dfa1c-271c-470a-aafa-09b455f97c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:18.520190 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dfa1c-271c-470a-aafa-09b455f97c05 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:18.520736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dfa1c-271c-470a-aafa-09b455f97c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:18.520736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dfa1c-271c-470a-aafa-09b455f97c05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:18.556944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a0e0b-4c5d-476f-8945-0b105964d1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:18.557758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a0e0b-4c5d-476f-8945-0b105964d1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:18.558043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a0e0b-4c5d-476f-8945-0b105964d1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:18.558336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a0e0b-4c5d-476f-8945-0b105964d1e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:18.581046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a0e0b-4c5d-476f-8945-0b105964d1e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-878a0e0b-4c5d-476f-8945-0b105964d1e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:18.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:18.589750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:18.590581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:18.591231 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 277/552] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:18.601743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a0e0b-4c5d-476f-8945-0b105964d1e4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:18.630186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7dfa1c-271c-470a-aafa-09b455f97c05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:18.631690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7dfa1c-271c-470a-aafa-09b455f97c05 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:18.726873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:18.739017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a0e0b-4c5d-476f-8945-0b105964d1e4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:18.771617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 276/553] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:18.773031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6685938-f387-4008-b7db-6b338b432bcf None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:18.773319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6685938-f387-4008-b7db-6b338b432bcf None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:18.773627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6685938-f387-4008-b7db-6b338b432bcf None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:18.774139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed540517-a9f5-4b79-b714-0568a0458cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:18.774890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed540517-a9f5-4b79-b714-0568a0458cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:18.775147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 277/554] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:18.775248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed540517-a9f5-4b79-b714-0568a0458cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:18.775451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed540517-a9f5-4b79-b714-0568a0458cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:18.776211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed540517-a9f5-4b79-b714-0568a0458cc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:18.776523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed540517-a9f5-4b79-b714-0568a0458cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:18.777128 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6685938-f387-4008-b7db-6b338b432bcf None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:18.777908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed540517-a9f5-4b79-b714-0568a0458cc0 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:18.790710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:18.796303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 278/555] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:18.797645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08210add-0dca-4220-8108-156dc64da118 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:18.797930 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08210add-0dca-4220-8108-156dc64da118 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:18.798213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08210add-0dca-4220-8108-156dc64da118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:18.819005 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-08210add-0dca-4220-8108-156dc64da118 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:18.820212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-08210add-0dca-4220-8108-156dc64da118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:18.898403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 279/556] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:18.898403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08210add-0dca-4220-8108-156dc64da118 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:18.904794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6685938-f387-4008-b7db-6b338b432bcf None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:18.961621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822cdae-6abe-4d2c-85c2-9367c88ec3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:18.962879 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822cdae-6abe-4d2c-85c2-9367c88ec3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:18.963393 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822cdae-6abe-4d2c-85c2-9367c88ec3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:18.963894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822cdae-6abe-4d2c-85c2-9367c88ec3d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:18.975815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822cdae-6abe-4d2c-85c2-9367c88ec3d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:18.977666 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822cdae-6abe-4d2c-85c2-9367c88ec3d2 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:18.982063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-9822cdae-6abe-4d2c-85c2-9367c88ec3d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:18.997414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:18.998376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:18.999268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:18.999268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:18.999268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:18.999268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:19.006083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08210add-0dca-4220-8108-156dc64da118 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:19.006810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 278/557] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:19.008186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822cdae-6abe-4d2c-85c2-9367c88ec3d2 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:19.051418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949292e4-635f-4ebe-a89e-871a3a67ff29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:19.052643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949292e4-635f-4ebe-a89e-871a3a67ff29 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.053170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949292e4-635f-4ebe-a89e-871a3a67ff29 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.053654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949292e4-635f-4ebe-a89e-871a3a67ff29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.068965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949292e4-635f-4ebe-a89e-871a3a67ff29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:19.071694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949292e4-635f-4ebe-a89e-871a3a67ff29 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:19.080911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-949292e4-635f-4ebe-a89e-871a3a67ff29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:19.102611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:19.104001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:19.104922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:19.104922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:19.104922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:19.104922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:19.112687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 280/558] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:19.112687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949292e4-635f-4ebe-a89e-871a3a67ff29 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:19.136817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822cdae-6abe-4d2c-85c2-9367c88ec3d2 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:19.172837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68510632-969a-47d6-9976-57c34a8cc513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:19.173719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68510632-969a-47d6-9976-57c34a8cc513 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.174354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68510632-969a-47d6-9976-57c34a8cc513 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.174636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68510632-969a-47d6-9976-57c34a8cc513 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.185007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68510632-969a-47d6-9976-57c34a8cc513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-68510632-969a-47d6-9976-57c34a8cc513 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:19.192654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:19.193579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:19.194837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 279/559] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:19.195617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68510632-969a-47d6-9976-57c34a8cc513 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:19.224845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949292e4-635f-4ebe-a89e-871a3a67ff29 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:19.261514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efcd50-7972-40e5-aa0d-0f81496a099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:19.262176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efcd50-7972-40e5-aa0d-0f81496a099e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.262485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efcd50-7972-40e5-aa0d-0f81496a099e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.262814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efcd50-7972-40e5-aa0d-0f81496a099e None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.263530 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efcd50-7972-40e5-aa0d-0f81496a099e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:19.263989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efcd50-7972-40e5-aa0d-0f81496a099e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:19.265582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efcd50-7972-40e5-aa0d-0f81496a099e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:19.267045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:19.282596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 281/560] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:19.283862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e60b-3a38-4c65-ba72-832ff53bd293 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.284148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e60b-3a38-4c65-ba72-832ff53bd293 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.284422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e60b-3a38-4c65-ba72-832ff53bd293 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.289693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 282/561] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:19.289693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613e60b-3a38-4c65-ba72-832ff53bd293 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:19.306828 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68510632-969a-47d6-9976-57c34a8cc513 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:19.331400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:19.344356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf796f-8d83-4e85-98a6-f3cf86d37908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:19.345027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf796f-8d83-4e85-98a6-f3cf86d37908 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.345321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf796f-8d83-4e85-98a6-f3cf86d37908 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.345602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf796f-8d83-4e85-98a6-f3cf86d37908 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.346355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf796f-8d83-4e85-98a6-f3cf86d37908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:19.346667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf796f-8d83-4e85-98a6-f3cf86d37908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:19.348034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf796f-8d83-4e85-98a6-f3cf86d37908 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:19.360695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 280/562] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:19.361468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf796f-8d83-4e85-98a6-f3cf86d37908 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:19.398842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613e60b-3a38-4c65-ba72-832ff53bd293 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:19.438655 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22267e6f-409f-4a27-93c2-6a0588ba529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:19.439246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22267e6f-409f-4a27-93c2-6a0588ba529b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.439532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22267e6f-409f-4a27-93c2-6a0588ba529b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.439812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22267e6f-409f-4a27-93c2-6a0588ba529b None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.440555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22267e6f-409f-4a27-93c2-6a0588ba529b None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:19.441063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22267e6f-409f-4a27-93c2-6a0588ba529b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:19.442230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22267e6f-409f-4a27-93c2-6a0588ba529b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:19.450584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 283/563] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:19.451681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be5cc-b533-436f-8d6c-27d17faeabfa None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.451969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be5cc-b533-436f-8d6c-27d17faeabfa None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.452244 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be5cc-b533-436f-8d6c-27d17faeabfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.469327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7d0be5cc-b533-436f-8d6c-27d17faeabfa None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:19.470541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7d0be5cc-b533-436f-8d6c-27d17faeabfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:19.470851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf796f-8d83-4e85-98a6-f3cf86d37908 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:19.506472 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b2f91-6b37-493a-a631-546a7ee66e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:19.507276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b2f91-6b37-493a-a631-546a7ee66e45 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.507555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b2f91-6b37-493a-a631-546a7ee66e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.507842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b2f91-6b37-493a-a631-546a7ee66e45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.514836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b2f91-6b37-493a-a631-546a7ee66e45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:19.516790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b2f91-6b37-493a-a631-546a7ee66e45 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:19.521954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-1b9b2f91-6b37-493a-a631-546a7ee66e45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:19.530653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:19.531877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:19.532780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:19.532780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:19.532780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:19.532780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:19.532780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:19.532780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:19.532780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:19.532780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:19.532780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 281/564] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:19.533261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b2f91-6b37-493a-a631-546a7ee66e45 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:19.554558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 284/565] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:19.554558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0be5cc-b533-436f-8d6c-27d17faeabfa None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:19.577679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:19.636809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b2f91-6b37-493a-a631-546a7ee66e45 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:19.663027 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0be5cc-b533-436f-8d6c-27d17faeabfa None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:19.672420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5e275-c4c8-45d4-9b79-680df5eeb149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:19.673225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5e275-c4c8-45d4-9b79-680df5eeb149 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.673511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5e275-c4c8-45d4-9b79-680df5eeb149 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.673782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5e275-c4c8-45d4-9b79-680df5eeb149 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.681246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5e275-c4c8-45d4-9b79-680df5eeb149 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:19.682728 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5e275-c4c8-45d4-9b79-680df5eeb149 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-aae5e275-c4c8-45d4-9b79-680df5eeb149 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:19.697302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:19.698383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:19.699359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:19.699359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:19.699359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:19.699359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:19.699359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:19.699359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:19.699359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:19.699359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:19.699359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 12:46:19.699359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:19.701669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597218b7-4104-4674-8118-211a8ed09b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:19.702486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597218b7-4104-4674-8118-211a8ed09b0b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.702949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597218b7-4104-4674-8118-211a8ed09b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.703052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 282/566] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:19.703141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597218b7-4104-4674-8118-211a8ed09b0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.703835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5e275-c4c8-45d4-9b79-680df5eeb149 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:19.713792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597218b7-4104-4674-8118-211a8ed09b0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:19.715283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597218b7-4104-4674-8118-211a8ed09b0b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-597218b7-4104-4674-8118-211a8ed09b0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:19.733758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:19.736126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:19.736126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:19.736126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:19.736126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:19.736126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 285/567] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:19.736126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597218b7-4104-4674-8118-211a8ed09b0b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:19.778912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:19.820330 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5e275-c4c8-45d4-9b79-680df5eeb149 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:19.821993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:19.844681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597218b7-4104-4674-8118-211a8ed09b0b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:19.857532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f974a49-ce82-4407-9dc9-6410f3e38643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:19.858495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f974a49-ce82-4407-9dc9-6410f3e38643 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.858900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f974a49-ce82-4407-9dc9-6410f3e38643 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.859332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f974a49-ce82-4407-9dc9-6410f3e38643 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.860371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f974a49-ce82-4407-9dc9-6410f3e38643 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:19.860783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f974a49-ce82-4407-9dc9-6410f3e38643 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:19.862325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f974a49-ce82-4407-9dc9-6410f3e38643 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:19.874538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 283/568] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:19.875975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f974a49-ce82-4407-9dc9-6410f3e38643 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:19.885485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732691d0-c769-45b9-8883-c6ea57fb99ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:19.887328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732691d0-c769-45b9-8883-c6ea57fb99ac None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:19.888186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732691d0-c769-45b9-8883-c6ea57fb99ac None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:19.888668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732691d0-c769-45b9-8883-c6ea57fb99ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:19.909095 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732691d0-c769-45b9-8883-c6ea57fb99ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-732691d0-c769-45b9-8883-c6ea57fb99ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:19.918170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:19.919097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 286/569] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:19.919945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732691d0-c769-45b9-8883-c6ea57fb99ac None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.005474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f974a49-ce82-4407-9dc9-6410f3e38643 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:20.026180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:20.030269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732691d0-c769-45b9-8883-c6ea57fb99ac None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:20.045616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895d41a-008d-4bfc-b3c4-34de4d1d03d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:20.046500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895d41a-008d-4bfc-b3c4-34de4d1d03d1 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.046925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895d41a-008d-4bfc-b3c4-34de4d1d03d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.047214 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895d41a-008d-4bfc-b3c4-34de4d1d03d1 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.048050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895d41a-008d-4bfc-b3c4-34de4d1d03d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:20.048356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895d41a-008d-4bfc-b3c4-34de4d1d03d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:20.049772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895d41a-008d-4bfc-b3c4-34de4d1d03d1 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:20.065102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe24af3-22eb-43d8-8a4f-9c669dc404ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:20.065775 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe24af3-22eb-43d8-8a4f-9c669dc404ed None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.066065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe24af3-22eb-43d8-8a4f-9c669dc404ed None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.066371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe24af3-22eb-43d8-8a4f-9c669dc404ed None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe24af3-22eb-43d8-8a4f-9c669dc404ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:20.067415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe24af3-22eb-43d8-8a4f-9c669dc404ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:20.068796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe24af3-22eb-43d8-8a4f-9c669dc404ed None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:20.070144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 284/570] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 12:46:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:20.070984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895d41a-008d-4bfc-b3c4-34de4d1d03d1 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.086225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 287/571] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:20.087327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe24af3-22eb-43d8-8a4f-9c669dc404ed None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.097448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:20.188523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895d41a-008d-4bfc-b3c4-34de4d1d03d1 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:20.190208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe24af3-22eb-43d8-8a4f-9c669dc404ed None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:20.221528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b75cb1-9a4c-4c82-80b1-ecbe3c9b2672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:20.222193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b75cb1-9a4c-4c82-80b1-ecbe3c9b2672 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.222506 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b75cb1-9a4c-4c82-80b1-ecbe3c9b2672 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.222786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b75cb1-9a4c-4c82-80b1-ecbe3c9b2672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.223380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0d940-49ea-4c0a-b2ed-94962f8880a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:20.223612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b75cb1-9a4c-4c82-80b1-ecbe3c9b2672 None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=d00b728319304b1e9ba2f1ee93022187) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:20.223925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b75cb1-9a4c-4c82-80b1-ecbe3c9b2672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:20.224069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0d940-49ea-4c0a-b2ed-94962f8880a5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.224330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0d940-49ea-4c0a-b2ed-94962f8880a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.224604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0d940-49ea-4c0a-b2ed-94962f8880a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.225342 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0d940-49ea-4c0a-b2ed-94962f8880a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:20.225430 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b75cb1-9a4c-4c82-80b1-ecbe3c9b2672 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:20.225650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0d940-49ea-4c0a-b2ed-94962f8880a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:20.227052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0d940-49ea-4c0a-b2ed-94962f8880a5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:20.230742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 285/572] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:20 2022] GET /identity/v3/role_assignments?user.id=d00b728319304b1e9ba2f1ee93022187&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:20.231505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b75cb1-9a4c-4c82-80b1-ecbe3c9b2672 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.246747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 288/573] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:20.247493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0d940-49ea-4c0a-b2ed-94962f8880a5 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.274050 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:20.329142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:20.347096 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b75cb1-9a4c-4c82-80b1-ecbe3c9b2672 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:20.364121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0d940-49ea-4c0a-b2ed-94962f8880a5 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:20.383818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d0527-e241-4f01-bdb0-f271ee7d7d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:20.384646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d0527-e241-4f01-bdb0-f271ee7d7d18 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.384933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d0527-e241-4f01-bdb0-f271ee7d7d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.385207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d0527-e241-4f01-bdb0-f271ee7d7d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.400590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d0527-e241-4f01-bdb0-f271ee7d7d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:20.402496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d0527-e241-4f01-bdb0-f271ee7d7d18 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:20.408997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f2857-c3b0-4253-b330-cae4081735e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:20.409828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f2857-c3b0-4253-b330-cae4081735e7 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.410109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f2857-c3b0-4253-b330-cae4081735e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.410435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f2857-c3b0-4253-b330-cae4081735e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.419124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f2857-c3b0-4253-b330-cae4081735e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:20.420679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f2857-c3b0-4253-b330-cae4081735e7 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-1d8d0527-e241-4f01-bdb0-f271ee7d7d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:20.422669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:20.423526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:20.424410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:20.424410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:20.424410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:20.424410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:20.424410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 286/574] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:20.425276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdca130-c2d6-4943-bda9-e2747d6c5730 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.425444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdca130-c2d6-4943-bda9-e2747d6c5730 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.425778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdca130-c2d6-4943-bda9-e2747d6c5730 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.427559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 287/575] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:20.427941 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdca130-c2d6-4943-bda9-e2747d6c5730 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-238f2857-c3b0-4253-b330-cae4081735e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:20.442738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:20.443720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 289/576] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 12:46:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:20.445168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f2857-c3b0-4253-b330-cae4081735e7 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.527990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:20.544904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdca130-c2d6-4943-bda9-e2747d6c5730 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:20.546691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f2857-c3b0-4253-b330-cae4081735e7 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:20.581044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54a56d-a25a-439b-867f-1fe6837ac2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:20.581720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54a56d-a25a-439b-867f-1fe6837ac2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.582011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54a56d-a25a-439b-867f-1fe6837ac2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.582275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024d0d9-796a-42b5-abee-734acc6578b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:20.582403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54a56d-a25a-439b-867f-1fe6837ac2a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.583043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54a56d-a25a-439b-867f-1fe6837ac2a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:20.583138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024d0d9-796a-42b5-abee-734acc6578b0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.583353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54a56d-a25a-439b-867f-1fe6837ac2a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:20.583448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024d0d9-796a-42b5-abee-734acc6578b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.583622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024d0d9-796a-42b5-abee-734acc6578b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.584724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54a56d-a25a-439b-867f-1fe6837ac2a0 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:20.586473 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:20.596205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024d0d9-796a-42b5-abee-734acc6578b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:20.602114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 290/577] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:20.602246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024d0d9-796a-42b5-abee-734acc6578b0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:20.602355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf648c37-38bb-4040-b07c-f418484cf308 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.602355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf648c37-38bb-4040-b07c-f418484cf308 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.602355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf648c37-38bb-4040-b07c-f418484cf308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.616835 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-bf648c37-38bb-4040-b07c-f418484cf308 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:20.618025 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bf648c37-38bb-4040-b07c-f418484cf308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-1024d0d9-796a-42b5-abee-734acc6578b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:20.620443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:20.621325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:20.622221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:20.622221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:20.622221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:20.622221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:20.622221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 288/578] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:20.622563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024d0d9-796a-42b5-abee-734acc6578b0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.694161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 291/579] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:20.695648 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf648c37-38bb-4040-b07c-f418484cf308 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.742398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024d0d9-796a-42b5-abee-734acc6578b0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:20.776879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:20.779904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a53bc-6c60-4d9c-9691-9bec1d5e7dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:20.781552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a53bc-6c60-4d9c-9691-9bec1d5e7dda None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.781836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a53bc-6c60-4d9c-9691-9bec1d5e7dda None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.782109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a53bc-6c60-4d9c-9691-9bec1d5e7dda None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.782976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a53bc-6c60-4d9c-9691-9bec1d5e7dda None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:20.783284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a53bc-6c60-4d9c-9691-9bec1d5e7dda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:20.784714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a53bc-6c60-4d9c-9691-9bec1d5e7dda None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:20.807807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 289/580] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 12:46:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:20.807807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a53bc-6c60-4d9c-9691-9bec1d5e7dda None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.816666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf648c37-38bb-4040-b07c-f418484cf308 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:20.852727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:20.853856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a505f1-d4d6-463c-95e3-1e91036687f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:20.854961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a505f1-d4d6-463c-95e3-1e91036687f3 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.855255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a505f1-d4d6-463c-95e3-1e91036687f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.855660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a505f1-d4d6-463c-95e3-1e91036687f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.862925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a505f1-d4d6-463c-95e3-1e91036687f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:20.864384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a505f1-d4d6-463c-95e3-1e91036687f3 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-78a505f1-d4d6-463c-95e3-1e91036687f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:20.878848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:20.879747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:20.880725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:20.880725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:20.880725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:20.880725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:20.880725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:20.880725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:20.880725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:20.880725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:20.880725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 292/581] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:20.881220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a505f1-d4d6-463c-95e3-1e91036687f3 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.912900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a53bc-6c60-4d9c-9691-9bec1d5e7dda None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:20.949134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ffdce-9eba-453a-bca3-d98b733a7e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:20.949893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ffdce-9eba-453a-bca3-d98b733a7e04 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:20.950243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ffdce-9eba-453a-bca3-d98b733a7e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:20.950617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ffdce-9eba-453a-bca3-d98b733a7e04 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:20.951432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ffdce-9eba-453a-bca3-d98b733a7e04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:20.951809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ffdce-9eba-453a-bca3-d98b733a7e04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:20.954036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ffdce-9eba-453a-bca3-d98b733a7e04 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:20.970510 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 290/582] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:20.970510 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ffdce-9eba-453a-bca3-d98b733a7e04 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:20.993966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a505f1-d4d6-463c-95e3-1e91036687f3 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:21.022224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:21.029287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1029793-3322-449d-ac56-3c7205fede03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:21.030212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1029793-3322-449d-ac56-3c7205fede03 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:21.030551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1029793-3322-449d-ac56-3c7205fede03 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:21.030891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1029793-3322-449d-ac56-3c7205fede03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:21.042842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1029793-3322-449d-ac56-3c7205fede03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-b1029793-3322-449d-ac56-3c7205fede03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:21.051070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:21.052020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 293/583] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:21.058183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1029793-3322-449d-ac56-3c7205fede03 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:21.079502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ffdce-9eba-453a-bca3-d98b733a7e04 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:21.087671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:21.117729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f7db1-f7ca-46aa-bd8e-bde7b993a981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:21.118554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f7db1-f7ca-46aa-bd8e-bde7b993a981 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:21.118839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f7db1-f7ca-46aa-bd8e-bde7b993a981 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:21.119110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f7db1-f7ca-46aa-bd8e-bde7b993a981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:21.129888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f7db1-f7ca-46aa-bd8e-bde7b993a981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:21.131352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f7db1-f7ca-46aa-bd8e-bde7b993a981 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-ac6f7db1-f7ca-46aa-bd8e-bde7b993a981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:21.150182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:21.151290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:21.157238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:21.157238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:21.157238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:21.157238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:21.157238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 291/584] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:21.157238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f7db1-f7ca-46aa-bd8e-bde7b993a981 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:21.163784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1029793-3322-449d-ac56-3c7205fede03 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:21.197921 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89115a5d-3ef8-418d-b49f-aaf254845a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:21.198731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89115a5d-3ef8-418d-b49f-aaf254845a87 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:21.199112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89115a5d-3ef8-418d-b49f-aaf254845a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:21.199490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89115a5d-3ef8-418d-b49f-aaf254845a87 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:21.200380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89115a5d-3ef8-418d-b49f-aaf254845a87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:21.200810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89115a5d-3ef8-418d-b49f-aaf254845a87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:21.202259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89115a5d-3ef8-418d-b49f-aaf254845a87 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:21.214833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 294/585] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:21.215770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89115a5d-3ef8-418d-b49f-aaf254845a87 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:21.264187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f7db1-f7ca-46aa-bd8e-bde7b993a981 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:21.270232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:21.298987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c6bca-7471-4ac7-a579-d25deba814cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:21.300032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c6bca-7471-4ac7-a579-d25deba814cb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:21.300319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c6bca-7471-4ac7-a579-d25deba814cb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:21.300593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c6bca-7471-4ac7-a579-d25deba814cb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:21.308285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c6bca-7471-4ac7-a579-d25deba814cb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:21.309733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c6bca-7471-4ac7-a579-d25deba814cb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-996c6bca-7471-4ac7-a579-d25deba814cb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:21.324801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:21.326027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:21.332257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:21.332257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:21.332257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:21.332257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:21.332257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:21.332257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:21.332257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:21.332257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:21.332257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 12:46:21.332257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:21.332257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 292/586] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:21.332981 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89115a5d-3ef8-418d-b49f-aaf254845a87 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:21.333075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c6bca-7471-4ac7-a579-d25deba814cb None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:21.352272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:21.370945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82f40d-f105-4989-a0f1-8b0afb5deeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:21.371616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82f40d-f105-4989-a0f1-8b0afb5deeba None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:21.371900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82f40d-f105-4989-a0f1-8b0afb5deeba None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:21.372174 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82f40d-f105-4989-a0f1-8b0afb5deeba None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:21.372918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82f40d-f105-4989-a0f1-8b0afb5deeba None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:21.373226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82f40d-f105-4989-a0f1-8b0afb5deeba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:21.374621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82f40d-f105-4989-a0f1-8b0afb5deeba None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:21.384765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 295/587] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:21.384765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82f40d-f105-4989-a0f1-8b0afb5deeba None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:21.449879 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c6bca-7471-4ac7-a579-d25deba814cb None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:21.486411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3750f90-021c-4108-89aa-3d6a61e4ac7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:21.487090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3750f90-021c-4108-89aa-3d6a61e4ac7d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:21.487375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3750f90-021c-4108-89aa-3d6a61e4ac7d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:21.487648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3750f90-021c-4108-89aa-3d6a61e4ac7d None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:21.488378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3750f90-021c-4108-89aa-3d6a61e4ac7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:21.488680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3750f90-021c-4108-89aa-3d6a61e4ac7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:21.490130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3750f90-021c-4108-89aa-3d6a61e4ac7d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:21.494371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82f40d-f105-4989-a0f1-8b0afb5deeba None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:21.507898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 293/588] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:21.507898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3750f90-021c-4108-89aa-3d6a61e4ac7d None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:21.509484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:21.531933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe84f3-2aa3-4ab3-ba2b-4c91bb87320c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:21.532912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe84f3-2aa3-4ab3-ba2b-4c91bb87320c None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:21.533256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe84f3-2aa3-4ab3-ba2b-4c91bb87320c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:21.533630 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe84f3-2aa3-4ab3-ba2b-4c91bb87320c None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/e47f1d65687e4cfcbce764076141702c/roles/1ea1fd1a01f6487aac4236749de7d4e1` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:21.550169 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe84f3-2aa3-4ab3-ba2b-4c91bb87320c None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=e47f1d65687e4cfcbce764076141702c, role_id=1ea1fd1a01f6487aac4236749de7d4e1)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:21.551660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe84f3-2aa3-4ab3-ba2b-4c91bb87320c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:21.565644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 296/589] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:46:21 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/e47f1d65687e4cfcbce764076141702c/roles/1ea1fd1a01f6487aac4236749de7d4e1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:46:21.566506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe84f3-2aa3-4ab3-ba2b-4c91bb87320c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:21.612877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:21.640269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3750f90-021c-4108-89aa-3d6a61e4ac7d None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:21.680134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43bd02-b3e3-49c8-bd34-832cbc70351e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:21.682652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43bd02-b3e3-49c8-bd34-832cbc70351e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:21.682652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43bd02-b3e3-49c8-bd34-832cbc70351e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:21.682652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43bd02-b3e3-49c8-bd34-832cbc70351e None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:21.682923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43bd02-b3e3-49c8-bd34-832cbc70351e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:21.683709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43bd02-b3e3-49c8-bd34-832cbc70351e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:21.685688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43bd02-b3e3-49c8-bd34-832cbc70351e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:21.705309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe84f3-2aa3-4ab3-ba2b-4c91bb87320c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:21.712967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 294/590] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 12:46:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:21.714271 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43bd02-b3e3-49c8-bd34-832cbc70351e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:21.746376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541483a-66eb-4256-bc88-c41ed4745a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:21.747226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541483a-66eb-4256-bc88-c41ed4745a02 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:21.747523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541483a-66eb-4256-bc88-c41ed4745a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:21.747804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541483a-66eb-4256-bc88-c41ed4745a02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:21.760199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541483a-66eb-4256-bc88-c41ed4745a02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:21.761957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541483a-66eb-4256-bc88-c41ed4745a02 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-5541483a-66eb-4256-bc88-c41ed4745a02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:21.782634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:21.783647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:21.784589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:21.784589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:21.784589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:21.784589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:21.784589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 297/591] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:21.785550 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541483a-66eb-4256-bc88-c41ed4745a02 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:21.802122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:21.837517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43bd02-b3e3-49c8-bd34-832cbc70351e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:21.888720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8b588-9af3-46da-8958-a6c7ec606b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:21.890904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8b588-9af3-46da-8958-a6c7ec606b25 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:21.891650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8b588-9af3-46da-8958-a6c7ec606b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:21.892162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8b588-9af3-46da-8958-a6c7ec606b25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:21.896586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:21.904043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8b588-9af3-46da-8958-a6c7ec606b25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:21.906761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8b588-9af3-46da-8958-a6c7ec606b25 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:21.913835 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541483a-66eb-4256-bc88-c41ed4745a02 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-75f8b588-9af3-46da-8958-a6c7ec606b25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:21.927407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:21.928350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:21.929236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:21.929236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:21.929236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:21.929236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:21.929236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:21.929236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:21.929236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:21.929236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:21.932283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 295/592] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:21.933446 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8b588-9af3-46da-8958-a6c7ec606b25 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:21.952643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584171f-a0fd-4807-8766-ced0315177f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:21.953631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584171f-a0fd-4807-8766-ced0315177f6 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:21.954017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584171f-a0fd-4807-8766-ced0315177f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:21.954431 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584171f-a0fd-4807-8766-ced0315177f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:21.965684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584171f-a0fd-4807-8766-ced0315177f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:21.967389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584171f-a0fd-4807-8766-ced0315177f6 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-6584171f-a0fd-4807-8766-ced0315177f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:21.987433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:21.988574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:21.989704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:21.989704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:21.989704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:21.989704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:21.991251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 298/593] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:21.992236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584171f-a0fd-4807-8766-ced0315177f6 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:22.063595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:22.072876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8b588-9af3-46da-8958-a6c7ec606b25 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:22.109518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d7dec-a4d1-4f82-af93-ab64df8788f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:22.110389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d7dec-a4d1-4f82-af93-ab64df8788f7 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.110771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d7dec-a4d1-4f82-af93-ab64df8788f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.111141 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d7dec-a4d1-4f82-af93-ab64df8788f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.111733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584171f-a0fd-4807-8766-ced0315177f6 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:22.112054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d7dec-a4d1-4f82-af93-ab64df8788f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:22.112454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d7dec-a4d1-4f82-af93-ab64df8788f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:22.114244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d7dec-a4d1-4f82-af93-ab64df8788f7 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:22.127878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 296/594] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:22.128650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d7dec-a4d1-4f82-af93-ab64df8788f7 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:22.151159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d9569-00a1-4264-96d0-514197f9379c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:22.152075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d9569-00a1-4264-96d0-514197f9379c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.152531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d9569-00a1-4264-96d0-514197f9379c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.152964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d9569-00a1-4264-96d0-514197f9379c None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.153901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d9569-00a1-4264-96d0-514197f9379c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:22.154398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d9569-00a1-4264-96d0-514197f9379c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:22.156534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d9569-00a1-4264-96d0-514197f9379c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:22.174339 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 299/595] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:22.175538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d9569-00a1-4264-96d0-514197f9379c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:22.195933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:22.241522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d7dec-a4d1-4f82-af93-ab64df8788f7 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:22.279617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea43f4-46d2-4c9e-b9f4-42f50211652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:22.280316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea43f4-46d2-4c9e-b9f4-42f50211652e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.280602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea43f4-46d2-4c9e-b9f4-42f50211652e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.280877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea43f4-46d2-4c9e-b9f4-42f50211652e None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.282338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea43f4-46d2-4c9e-b9f4-42f50211652e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:22.282656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea43f4-46d2-4c9e-b9f4-42f50211652e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:22.284183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea43f4-46d2-4c9e-b9f4-42f50211652e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:22.287125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d9569-00a1-4264-96d0-514197f9379c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:22.297536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 297/596] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:22.298530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea43f4-46d2-4c9e-b9f4-42f50211652e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:22.317383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:22.327602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a85c1-645f-4105-929b-d37f92ee19ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:22.328757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a85c1-645f-4105-929b-d37f92ee19ce None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.329198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a85c1-645f-4105-929b-d37f92ee19ce None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.329626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a85c1-645f-4105-929b-d37f92ee19ce None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.338423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a85c1-645f-4105-929b-d37f92ee19ce None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:22.340077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a85c1-645f-4105-929b-d37f92ee19ce None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-f49a85c1-645f-4105-929b-d37f92ee19ce None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:22.355509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:22.356684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:22.357817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:22.357817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:22.357817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:22.357817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:22.357817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:22.357817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:22.357817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:22.357817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:22.357817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 12:46:22.357817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:22.360329 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 300/597] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:22.361481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a85c1-645f-4105-929b-d37f92ee19ce None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:22.425023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea43f4-46d2-4c9e-b9f4-42f50211652e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:22.446766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:22.469713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78180a1b-e5c8-43e4-8527-bd487d8f5847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:22.470640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180a1b-e5c8-43e4-8527-bd487d8f5847 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.470920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180a1b-e5c8-43e4-8527-bd487d8f5847 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.471192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180a1b-e5c8-43e4-8527-bd487d8f5847 None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/05b2edb6fcb7496bb55650080a9da172/roles/1ea1fd1a01f6487aac4236749de7d4e1` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.478061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a85c1-645f-4105-929b-d37f92ee19ce None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:22.486460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78180a1b-e5c8-43e4-8527-bd487d8f5847 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=05b2edb6fcb7496bb55650080a9da172, role_id=1ea1fd1a01f6487aac4236749de7d4e1)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:22.487906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78180a1b-e5c8-43e4-8527-bd487d8f5847 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:22.500646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 298/598] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:46:22 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/05b2edb6fcb7496bb55650080a9da172/roles/1ea1fd1a01f6487aac4236749de7d4e1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:46:22.501787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78180a1b-e5c8-43e4-8527-bd487d8f5847 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:22.529139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb2610-a121-45b6-9748-217b0bb796e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:22.531020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb2610-a121-45b6-9748-217b0bb796e0 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.531525 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb2610-a121-45b6-9748-217b0bb796e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.532007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb2610-a121-45b6-9748-217b0bb796e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.543778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb2610-a121-45b6-9748-217b0bb796e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:22.547078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb2610-a121-45b6-9748-217b0bb796e0 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-d6eb2610-a121-45b6-9748-217b0bb796e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:22.568081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:22.569051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:22.569961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:22.569961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:22.569961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:22.569961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:22.569961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:22.569961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:22.569961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:22.569961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:22.576523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 301/599] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:22.576523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd22635-fb61-4bdb-a9ba-09c7275c4251 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.576523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd22635-fb61-4bdb-a9ba-09c7275c4251 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.576523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd22635-fb61-4bdb-a9ba-09c7275c4251 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.582432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 302/600] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:22.584120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd22635-fb61-4bdb-a9ba-09c7275c4251 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:22.648991 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78180a1b-e5c8-43e4-8527-bd487d8f5847 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:22.666510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:22.695676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240caeb-df5c-4e6c-b38e-930700cc854c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:22.697469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240caeb-df5c-4e6c-b38e-930700cc854c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.697938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240caeb-df5c-4e6c-b38e-930700cc854c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.698417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240caeb-df5c-4e6c-b38e-930700cc854c None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.699265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240caeb-df5c-4e6c-b38e-930700cc854c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:22.699570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240caeb-df5c-4e6c-b38e-930700cc854c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:22.701084 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240caeb-df5c-4e6c-b38e-930700cc854c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:22.722310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 299/601] 10.222.0.45 () {56 vars in 1139 bytes} [Tue May 24 12:46:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:22.723707 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995e731-c50d-405d-b3d8-d051e3e61aaa None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.723976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995e731-c50d-405d-b3d8-d051e3e61aaa None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.724405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995e731-c50d-405d-b3d8-d051e3e61aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.742969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a995e731-c50d-405d-b3d8-d051e3e61aaa None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:22.744140 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a995e731-c50d-405d-b3d8-d051e3e61aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:22.750179 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd22635-fb61-4bdb-a9ba-09c7275c4251 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:22.763462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:22.802703 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d2618-797a-4496-b376-ced5e30335bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:22.803383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d2618-797a-4496-b376-ced5e30335bc None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.803668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d2618-797a-4496-b376-ced5e30335bc None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.803946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d2618-797a-4496-b376-ced5e30335bc None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.804784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d2618-797a-4496-b376-ced5e30335bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:22.805078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d2618-797a-4496-b376-ced5e30335bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:22.806593 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d2618-797a-4496-b376-ced5e30335bc None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:22.824514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 303/602] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:22.824514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d2618-797a-4496-b376-ced5e30335bc None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:22.838354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 300/603] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:22.839575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995e731-c50d-405d-b3d8-d051e3e61aaa None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:22.946064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d2618-797a-4496-b376-ced5e30335bc None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:22.950934 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995e731-c50d-405d-b3d8-d051e3e61aaa None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:22.953735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:22.982868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ae4f2-69b5-48ff-901d-3b939a5978c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:22.983726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ae4f2-69b5-48ff-901d-3b939a5978c5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.983961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ae4f2-69b5-48ff-901d-3b939a5978c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.984265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ae4f2-69b5-48ff-901d-3b939a5978c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.991657 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad83346-02c7-45ce-b0a2-18211d148de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:22.992905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad83346-02c7-45ce-b0a2-18211d148de7 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:22.993455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad83346-02c7-45ce-b0a2-18211d148de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:22.993947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad83346-02c7-45ce-b0a2-18211d148de7 None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/8c9f08515d514be8af1176717ec9e6fd/roles/1ea1fd1a01f6487aac4236749de7d4e1` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:22.995266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ae4f2-69b5-48ff-901d-3b939a5978c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:22.996932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ae4f2-69b5-48ff-901d-3b939a5978c5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:23.016851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad83346-02c7-45ce-b0a2-18211d148de7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=8c9f08515d514be8af1176717ec9e6fd, role_id=1ea1fd1a01f6487aac4236749de7d4e1)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:23.018502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad83346-02c7-45ce-b0a2-18211d148de7 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-891ae4f2-69b5-48ff-901d-3b939a5978c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:23.028585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:23.029864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:23.032206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:23.032206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:23.032206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:23.032206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:23.034202 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 304/604] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:23.035224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ae4f2-69b5-48ff-901d-3b939a5978c5 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:23.045387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 301/605] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:46:22 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/8c9f08515d514be8af1176717ec9e6fd/roles/1ea1fd1a01f6487aac4236749de7d4e1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:46:23.046385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad83346-02c7-45ce-b0a2-18211d148de7 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:23.060596 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:23.208452 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ae4f2-69b5-48ff-901d-3b939a5978c5 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:23.217803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad83346-02c7-45ce-b0a2-18211d148de7 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:23.276651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f9e37-d04d-4dbd-ad93-bab1933be04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:23.277509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f9e37-d04d-4dbd-ad93-bab1933be04b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:23.277795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f9e37-d04d-4dbd-ad93-bab1933be04b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:23.278067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f9e37-d04d-4dbd-ad93-bab1933be04b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:23.289850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f9e37-d04d-4dbd-ad93-bab1933be04b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-c11f9e37-d04d-4dbd-ad93-bab1933be04b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:23.298875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:23.299890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:23.301427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 305/606] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:23.305974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f9e37-d04d-4dbd-ad93-bab1933be04b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:23.307942 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426bc8-9efe-4fea-ab9e-4543dd0c4b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:23.309120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426bc8-9efe-4fea-ab9e-4543dd0c4b48 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:23.309615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426bc8-9efe-4fea-ab9e-4543dd0c4b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:23.311106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426bc8-9efe-4fea-ab9e-4543dd0c4b48 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:23.312003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46426bc8-9efe-4fea-ab9e-4543dd0c4b48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:23.312542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46426bc8-9efe-4fea-ab9e-4543dd0c4b48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:23.314112 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46426bc8-9efe-4fea-ab9e-4543dd0c4b48 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:23.328716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:23.331719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 302/607] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:23.339147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426bc8-9efe-4fea-ab9e-4543dd0c4b48 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:23.425950 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:23.427899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f9e37-d04d-4dbd-ad93-bab1933be04b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:23.466333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e962597-7551-4e20-b31e-5c1a8da0bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:23.467344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e962597-7551-4e20-b31e-5c1a8da0bb1a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:23.467874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e962597-7551-4e20-b31e-5c1a8da0bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:23.468962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e962597-7551-4e20-b31e-5c1a8da0bb1a None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:23.470013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e962597-7551-4e20-b31e-5c1a8da0bb1a None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:23.470617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e962597-7551-4e20-b31e-5c1a8da0bb1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:23.472418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e962597-7551-4e20-b31e-5c1a8da0bb1a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:23.475278 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426bc8-9efe-4fea-ab9e-4543dd0c4b48 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:23.481950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 306/608] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:23.488895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e962597-7551-4e20-b31e-5c1a8da0bb1a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:23.526207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90f381-6eab-43c1-9c46-34401b2710a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:23.527047 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f381-6eab-43c1-9c46-34401b2710a2 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:23.527334 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f381-6eab-43c1-9c46-34401b2710a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:23.527612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f381-6eab-43c1-9c46-34401b2710a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:23.547448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90f381-6eab-43c1-9c46-34401b2710a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:23.548912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90f381-6eab-43c1-9c46-34401b2710a2 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-6b90f381-6eab-43c1-9c46-34401b2710a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:23.586891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:23.587948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:23.588844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:23.588844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:23.588844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:23.588844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:23.588844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:23.588844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:23.588844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:23.588844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:23.588844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 303/609] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:23.604674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90f381-6eab-43c1-9c46-34401b2710a2 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:23.605547 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:23.627566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e962597-7551-4e20-b31e-5c1a8da0bb1a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:23.665711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5e0d4-2a66-4535-b041-995862ca7a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:23.666702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5e0d4-2a66-4535-b041-995862ca7a11 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:23.667003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5e0d4-2a66-4535-b041-995862ca7a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:23.667288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5e0d4-2a66-4535-b041-995862ca7a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:23.679857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5e0d4-2a66-4535-b041-995862ca7a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:23.681339 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5e0d4-2a66-4535-b041-995862ca7a11 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-afb5e0d4-2a66-4535-b041-995862ca7a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:23.700964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:23.701889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:23.702875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:23.702875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:23.702875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:23.702875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:23.702875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 307/610] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:23.707970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5e0d4-2a66-4535-b041-995862ca7a11 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:23.842058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5e0d4-2a66-4535-b041-995862ca7a11 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:23.864212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90f381-6eab-43c1-9c46-34401b2710a2 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:23.875867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:23.883721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484acc1-8224-4ec2-bf5a-31cb9e8846c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:23.884694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484acc1-8224-4ec2-bf5a-31cb9e8846c9 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:23.885198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484acc1-8224-4ec2-bf5a-31cb9e8846c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:23.885680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484acc1-8224-4ec2-bf5a-31cb9e8846c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:23.886891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484acc1-8224-4ec2-bf5a-31cb9e8846c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:23.887669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484acc1-8224-4ec2-bf5a-31cb9e8846c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:23.889559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484acc1-8224-4ec2-bf5a-31cb9e8846c9 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:23.904609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 308/611] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:23.910131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:23.910601 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484acc1-8224-4ec2-bf5a-31cb9e8846c9 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:23.940553 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132e4b-7274-448d-9bbd-5321ae1d0377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:23.941232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132e4b-7274-448d-9bbd-5321ae1d0377 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:23.941518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132e4b-7274-448d-9bbd-5321ae1d0377 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:23.941791 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132e4b-7274-448d-9bbd-5321ae1d0377 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:23.942608 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13132e4b-7274-448d-9bbd-5321ae1d0377 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:23.942916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13132e4b-7274-448d-9bbd-5321ae1d0377 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:23.944470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13132e4b-7274-448d-9bbd-5321ae1d0377 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:23.980489 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 304/612] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:23.983186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132e4b-7274-448d-9bbd-5321ae1d0377 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:24.112555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484acc1-8224-4ec2-bf5a-31cb9e8846c9 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:24.120280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132e4b-7274-448d-9bbd-5321ae1d0377 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:24.154729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115a269-c232-4cbc-9bbf-2e942fc4b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:24.155530 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115a269-c232-4cbc-9bbf-2e942fc4b848 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:24.155830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115a269-c232-4cbc-9bbf-2e942fc4b848 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:24.156647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115a269-c232-4cbc-9bbf-2e942fc4b848 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:24.159881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8133ff0-2335-4954-bbee-f7b340d6a464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:24.161036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8133ff0-2335-4954-bbee-f7b340d6a464 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:24.161535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8133ff0-2335-4954-bbee-f7b340d6a464 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:24.162015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8133ff0-2335-4954-bbee-f7b340d6a464 None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/bb111be6bc2946d3b50b8d99a89cf8ac/roles/1ea1fd1a01f6487aac4236749de7d4e1` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:24.164160 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115a269-c232-4cbc-9bbf-2e942fc4b848 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:24.167069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115a269-c232-4cbc-9bbf-2e942fc4b848 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:24.180307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8133ff0-2335-4954-bbee-f7b340d6a464 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=bb111be6bc2946d3b50b8d99a89cf8ac, role_id=1ea1fd1a01f6487aac4236749de7d4e1)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:24.182161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8133ff0-2335-4954-bbee-f7b340d6a464 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-2115a269-c232-4cbc-9bbf-2e942fc4b848 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:24.183990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:24.184970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:24.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:24.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:24.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:24.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:24.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:24.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:24.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:24.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:24.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:24.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:24.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 309/613] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:24.186966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a72f-07ce-4e93-acb8-fe37660ffce3 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:24.187036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a72f-07ce-4e93-acb8-fe37660ffce3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:24.187890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a72f-07ce-4e93-acb8-fe37660ffce3 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:24.191304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 310/614] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:24.193813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5a72f-07ce-4e93-acb8-fe37660ffce3 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:24.211101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 305/615] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:46:23 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/bb111be6bc2946d3b50b8d99a89cf8ac/roles/1ea1fd1a01f6487aac4236749de7d4e1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 12:46:24.215994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb50de-0688-4e4e-9f26-3a7d5c06eb48 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:24.215994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb50de-0688-4e4e-9f26-3a7d5c06eb48 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:24.215994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb50de-0688-4e4e-9f26-3a7d5c06eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:24.246887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-38fb50de-0688-4e4e-9f26-3a7d5c06eb48 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:24.248116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-38fb50de-0688-4e4e-9f26-3a7d5c06eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:24.304662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:24.313731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:24.361723 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5a72f-07ce-4e93-acb8-fe37660ffce3 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:24.383540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 306/616] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:24.418233 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2aaa8-2ffd-440e-a092-a9fe5211cf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:24.419090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2aaa8-2ffd-440e-a092-a9fe5211cf0b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:24.419463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2aaa8-2ffd-440e-a092-a9fe5211cf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:24.419747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2aaa8-2ffd-440e-a092-a9fe5211cf0b None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:24.420617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2aaa8-2ffd-440e-a092-a9fe5211cf0b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:24.420932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2aaa8-2ffd-440e-a092-a9fe5211cf0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:24.422519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2aaa8-2ffd-440e-a092-a9fe5211cf0b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:24.435389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb50de-0688-4e4e-9f26-3a7d5c06eb48 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:24.447342 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 311/617] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 12:46:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:24.454991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2aaa8-2ffd-440e-a092-a9fe5211cf0b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:24.562836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2aaa8-2ffd-440e-a092-a9fe5211cf0b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:24.573648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb50de-0688-4e4e-9f26-3a7d5c06eb48 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:24.574212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:24.598768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b80421-ba11-42d1-a86e-471e206e563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:24.599743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b80421-ba11-42d1-a86e-471e206e563a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:24.599856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b80421-ba11-42d1-a86e-471e206e563a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:24.600109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b80421-ba11-42d1-a86e-471e206e563a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:24.611395 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b80421-ba11-42d1-a86e-471e206e563a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:24.612919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b80421-ba11-42d1-a86e-471e206e563a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:24.616447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c56c4b-fc69-4c96-8fc8-ff85f7248bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:24.617534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c56c4b-fc69-4c96-8fc8-ff85f7248bcf None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:24.618033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c56c4b-fc69-4c96-8fc8-ff85f7248bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:24.618529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c56c4b-fc69-4c96-8fc8-ff85f7248bcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:24.629889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c56c4b-fc69-4c96-8fc8-ff85f7248bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-f4b80421-ba11-42d1-a86e-471e206e563a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:24.632500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:24.633402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:24.634350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:24.634350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:24.634350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:24.634350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:24.634350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 312/618] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:24.646420 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b80421-ba11-42d1-a86e-471e206e563a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-93c56c4b-fc69-4c96-8fc8-ff85f7248bcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:24.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:24.647488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:24.648320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 307/619] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:24.659307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c56c4b-fc69-4c96-8fc8-ff85f7248bcf None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:24.719643 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:24.815842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b80421-ba11-42d1-a86e-471e206e563a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:24.862811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c56c4b-fc69-4c96-8fc8-ff85f7248bcf None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:24.882164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8711b-8b5d-47a4-9845-13d7405bef82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:24.882871 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8711b-8b5d-47a4-9845-13d7405bef82 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:24.883128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8711b-8b5d-47a4-9845-13d7405bef82 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:24.883402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8711b-8b5d-47a4-9845-13d7405bef82 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:24.884123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8711b-8b5d-47a4-9845-13d7405bef82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:24.884480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8711b-8b5d-47a4-9845-13d7405bef82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:24.886039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8711b-8b5d-47a4-9845-13d7405bef82 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:24.904098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 313/620] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:24.911237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8711b-8b5d-47a4-9845-13d7405bef82 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:24.914580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e37f9-9545-4b1f-861b-7688e540892a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:24.915256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e37f9-9545-4b1f-861b-7688e540892a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:24.915538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e37f9-9545-4b1f-861b-7688e540892a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:24.915810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e37f9-9545-4b1f-861b-7688e540892a None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:24.916665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e37f9-9545-4b1f-861b-7688e540892a None admin] RBAC: Adding query filter params (name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:24.916972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e37f9-9545-4b1f-861b-7688e540892a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:24.918469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e37f9-9545-4b1f-861b-7688e540892a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:24.927510 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 308/621] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:24.932497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e37f9-9545-4b1f-861b-7688e540892a None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:24.955841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:25.053200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:25.068683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8711b-8b5d-47a4-9845-13d7405bef82 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:25.071229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e37f9-9545-4b1f-861b-7688e540892a None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:25.105886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038d9eb-0e48-44fe-b1fb-971de6c6ad7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:25.107177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038d9eb-0e48-44fe-b1fb-971de6c6ad7d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.107744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038d9eb-0e48-44fe-b1fb-971de6c6ad7d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.108230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038d9eb-0e48-44fe-b1fb-971de6c6ad7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.116300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c9de7-f5c9-4cd1-8daa-fe77fac137c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:25.117215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c9de7-f5c9-4cd1-8daa-fe77fac137c5 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.117450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038d9eb-0e48-44fe-b1fb-971de6c6ad7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:25.117554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c9de7-f5c9-4cd1-8daa-fe77fac137c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.117767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c9de7-f5c9-4cd1-8daa-fe77fac137c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.119779 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038d9eb-0e48-44fe-b1fb-971de6c6ad7d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:25.130172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c9de7-f5c9-4cd1-8daa-fe77fac137c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:25.131678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c9de7-f5c9-4cd1-8daa-fe77fac137c5 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-8038d9eb-0e48-44fe-b1fb-971de6c6ad7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:25.138518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:25.139424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:25.140410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:25.140410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:25.140410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:25.140410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:25.140410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:25.140410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:25.140410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:25.140410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:25.143900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 314/622] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:25.145448 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60031ed9-1e72-4c61-99a2-63f55fb02454 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.145962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60031ed9-1e72-4c61-99a2-63f55fb02454 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.146478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60031ed9-1e72-4c61-99a2-63f55fb02454 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.148393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 315/623] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:25.149347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60031ed9-1e72-4c61-99a2-63f55fb02454 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-5c4c9de7-f5c9-4cd1-8daa-fe77fac137c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:25.158257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:25.159256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:25.160151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:25.160151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:25.160151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:25.160151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:25.166267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 309/624] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:25.167272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c9de7-f5c9-4cd1-8daa-fe77fac137c5 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:25.250889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:25.273327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60031ed9-1e72-4c61-99a2-63f55fb02454 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:25.302558 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c9de7-f5c9-4cd1-8daa-fe77fac137c5 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:25.312371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b5d34-d191-44e4-8c16-264e38c687e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:25.313549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b5d34-d191-44e4-8c16-264e38c687e9 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.314130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b5d34-d191-44e4-8c16-264e38c687e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.314674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b5d34-d191-44e4-8c16-264e38c687e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.315785 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b5d34-d191-44e4-8c16-264e38c687e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:25.316304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b5d34-d191-44e4-8c16-264e38c687e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:25.318007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b5d34-d191-44e4-8c16-264e38c687e9 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:25.338427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 316/625] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:25.340153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3140c-d9c6-4905-989d-4b1b0fc052c3 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.340663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3140c-d9c6-4905-989d-4b1b0fc052c3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.341122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3140c-d9c6-4905-989d-4b1b0fc052c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.347835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc47f0b-2ab5-463d-a113-8665a636c2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:25.349409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc47f0b-2ab5-463d-a113-8665a636c2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.349769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc47f0b-2ab5-463d-a113-8665a636c2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.350117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc47f0b-2ab5-463d-a113-8665a636c2b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.350953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc47f0b-2ab5-463d-a113-8665a636c2b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:25.351327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc47f0b-2ab5-463d-a113-8665a636c2b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:25.352874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc47f0b-2ab5-463d-a113-8665a636c2b8 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:25.361182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:25.362643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-45c3140c-d9c6-4905-989d-4b1b0fc052c3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:25.364228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-45c3140c-d9c6-4905-989d-4b1b0fc052c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:25.369148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 310/626] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:25.370165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc47f0b-2ab5-463d-a113-8665a636c2b8 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:25.462525 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 317/627] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:25.463774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3140c-d9c6-4905-989d-4b1b0fc052c3 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:25.502139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc47f0b-2ab5-463d-a113-8665a636c2b8 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:25.540624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145749-f1ea-4f3f-9a57-9673e84c1bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:25.541690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145749-f1ea-4f3f-9a57-9673e84c1bd4 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.542082 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145749-f1ea-4f3f-9a57-9673e84c1bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.542496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145749-f1ea-4f3f-9a57-9673e84c1bd4 None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/d00b728319304b1e9ba2f1ee93022187/roles/1ea1fd1a01f6487aac4236749de7d4e1` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.557075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:25.558317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91145749-f1ea-4f3f-9a57-9673e84c1bd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=d00b728319304b1e9ba2f1ee93022187, role_id=1ea1fd1a01f6487aac4236749de7d4e1)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:25.559793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91145749-f1ea-4f3f-9a57-9673e84c1bd4 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:25.582772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 311/628] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:46:25 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/d00b728319304b1e9ba2f1ee93022187/roles/1ea1fd1a01f6487aac4236749de7d4e1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:46:25.583976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145749-f1ea-4f3f-9a57-9673e84c1bd4 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:25.591964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3140c-d9c6-4905-989d-4b1b0fc052c3 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:25.630955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16a2a2-ceae-4806-a247-2cc16ac595ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:25.632123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16a2a2-ceae-4806-a247-2cc16ac595ba None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.632565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16a2a2-ceae-4806-a247-2cc16ac595ba None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.633014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16a2a2-ceae-4806-a247-2cc16ac595ba None admin] PATH_INFO: `/v3/users/glance` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.642741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16a2a2-ceae-4806-a247-2cc16ac595ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:25.644764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16a2a2-ceae-4806-a247-2cc16ac595ba None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:25.679654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-bd16a2a2-ceae-4806-a247-2cc16ac595ba None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:25.680750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:25.681787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:25.682818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:25.682818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:25.682818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:25.682818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:25.682818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:25.682818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:25.682818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:25.682818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:25.682818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 12:46:25.682818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:25.682818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 318/629] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:25.696129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae622a8-6c1c-4b8f-96e1-e0acc6320a28 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.696447 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae622a8-6c1c-4b8f-96e1-e0acc6320a28 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.696737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae622a8-6c1c-4b8f-96e1-e0acc6320a28 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.698326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 319/630] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:25.698973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae622a8-6c1c-4b8f-96e1-e0acc6320a28 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:25.736233 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145749-f1ea-4f3f-9a57-9673e84c1bd4 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:25.781801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d07630-5105-4a58-99e5-9d24746acb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:25.783143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d07630-5105-4a58-99e5-9d24746acb0d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.783711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d07630-5105-4a58-99e5-9d24746acb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.784191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d07630-5105-4a58-99e5-9d24746acb0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.797371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d07630-5105-4a58-99e5-9d24746acb0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-45d07630-5105-4a58-99e5-9d24746acb0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:25.806048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:25.807094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:25.807921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:25.811925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 312/631] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:25.813613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133a539-8a90-4596-ae9d-40536bb460cd None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.814125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133a539-8a90-4596-ae9d-40536bb460cd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.814653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133a539-8a90-4596-ae9d-40536bb460cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.832664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-6133a539-8a90-4596-ae9d-40536bb460cd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:25.833974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae622a8-6c1c-4b8f-96e1-e0acc6320a28 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:25.835202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6133a539-8a90-4596-ae9d-40536bb460cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:25.867739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:25.873569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd3f08-5ae7-471b-b6a0-392f272b9378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:25.875011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd3f08-5ae7-471b-b6a0-392f272b9378 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:25.875011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd3f08-5ae7-471b-b6a0-392f272b9378 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:25.875011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd3f08-5ae7-471b-b6a0-392f272b9378 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:25.875819 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd3f08-5ae7-471b-b6a0-392f272b9378 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:25.876134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd3f08-5ae7-471b-b6a0-392f272b9378 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:25.877577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd3f08-5ae7-471b-b6a0-392f272b9378 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:25.899707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 320/632] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 12:46:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:25.900552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd3f08-5ae7-471b-b6a0-392f272b9378 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:25.930498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 313/633] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:25.930498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133a539-8a90-4596-ae9d-40536bb460cd None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:25.989322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:26.043881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd3f08-5ae7-471b-b6a0-392f272b9378 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:26.057823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133a539-8a90-4596-ae9d-40536bb460cd None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:26.088186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f4cce-7da9-41ff-ad62-f6090f7c7068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:26.089119 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f4cce-7da9-41ff-ad62-f6090f7c7068 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:26.089559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f4cce-7da9-41ff-ad62-f6090f7c7068 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:26.089968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f4cce-7da9-41ff-ad62-f6090f7c7068 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:26.090962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f4cce-7da9-41ff-ad62-f6090f7c7068 None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:26.091400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f4cce-7da9-41ff-ad62-f6090f7c7068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:26.093019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f4cce-7da9-41ff-ad62-f6090f7c7068 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:26.100490 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80db80-8543-48c9-910b-95ba2426b4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:26.101323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80db80-8543-48c9-910b-95ba2426b4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:26.101738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80db80-8543-48c9-910b-95ba2426b4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:26.102127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80db80-8543-48c9-910b-95ba2426b4b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:26.103171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 321/634] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:26.104256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f4cce-7da9-41ff-ad62-f6090f7c7068 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:26.173068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:26.205686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80db80-8543-48c9-910b-95ba2426b4b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:26.205686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80db80-8543-48c9-910b-95ba2426b4b5 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:26.231626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f4cce-7da9-41ff-ad62-f6090f7c7068 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:26.252650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:26.277912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402eba3-5cc3-4ff6-8a23-1f78290c0685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:26.279668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402eba3-5cc3-4ff6-8a23-1f78290c0685 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:26.280121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402eba3-5cc3-4ff6-8a23-1f78290c0685 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:26.280543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402eba3-5cc3-4ff6-8a23-1f78290c0685 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:26.294569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402eba3-5cc3-4ff6-8a23-1f78290c0685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-5402eba3-5cc3-4ff6-8a23-1f78290c0685 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:26.305333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:26.306462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:26.307464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:26.310337 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 322/635] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:26.311324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402eba3-5cc3-4ff6-8a23-1f78290c0685 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:26.339859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 314/636] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:26.339859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80db80-8543-48c9-910b-95ba2426b4b5 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:26.442235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402eba3-5cc3-4ff6-8a23-1f78290c0685 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:26.457255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:26.478213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80db80-8543-48c9-910b-95ba2426b4b5 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:26.483462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b138aa-e4fa-4701-ae51-54deaebd4cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:26.484439 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b138aa-e4fa-4701-ae51-54deaebd4cad None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:26.484834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b138aa-e4fa-4701-ae51-54deaebd4cad None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:26.485223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b138aa-e4fa-4701-ae51-54deaebd4cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:26.496645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b138aa-e4fa-4701-ae51-54deaebd4cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:26.498269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b138aa-e4fa-4701-ae51-54deaebd4cad None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:26.517130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4f440-4995-44ba-8684-3940c60f17f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-a0b138aa-e4fa-4701-ae51-54deaebd4cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:26.517585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:26.518672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:26.519698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:26.519698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:26.519698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:26.519698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:26.520431 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4f440-4995-44ba-8684-3940c60f17f0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:26.520851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4f440-4995-44ba-8684-3940c60f17f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:26.525455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4f440-4995-44ba-8684-3940c60f17f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:26.525583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 323/637] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:26.525583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b138aa-e4fa-4701-ae51-54deaebd4cad None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:26.526793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4f440-4995-44ba-8684-3940c60f17f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:26.527314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4f440-4995-44ba-8684-3940c60f17f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:26.528826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4f440-4995-44ba-8684-3940c60f17f0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:26.534767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:26.540286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 315/638] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:26.541388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4f440-4995-44ba-8684-3940c60f17f0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:26.653578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b138aa-e4fa-4701-ae51-54deaebd4cad None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:26.675949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4f440-4995-44ba-8684-3940c60f17f0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:26.699371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017907-3b23-4e24-ac8f-4fa24350ca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:26.700393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017907-3b23-4e24-ac8f-4fa24350ca57 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:26.700827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017907-3b23-4e24-ac8f-4fa24350ca57 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:26.701781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017907-3b23-4e24-ac8f-4fa24350ca57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:26.716367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd017907-3b23-4e24-ac8f-4fa24350ca57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:26.717695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03e9d5-8ac9-451c-9d81-72da5adb26f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:26.718935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd017907-3b23-4e24-ac8f-4fa24350ca57 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:26.719220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e9d5-8ac9-451c-9d81-72da5adb26f3 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:26.719611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e9d5-8ac9-451c-9d81-72da5adb26f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:26.720192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e9d5-8ac9-451c-9d81-72da5adb26f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:26.721876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03e9d5-8ac9-451c-9d81-72da5adb26f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:26.721876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03e9d5-8ac9-451c-9d81-72da5adb26f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:26.723077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03e9d5-8ac9-451c-9d81-72da5adb26f3 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:26.742996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 316/639] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:26.742996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03e9d5-8ac9-451c-9d81-72da5adb26f3 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-dd017907-3b23-4e24-ac8f-4fa24350ca57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:26.751090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:26.752191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:26.753248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:26.753248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:26.753248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:26.753248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:26.759919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 324/640] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:26.759919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017907-3b23-4e24-ac8f-4fa24350ca57 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:26.780234 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:26.824573 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:26.868986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017907-3b23-4e24-ac8f-4fa24350ca57 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:26.869477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03e9d5-8ac9-451c-9d81-72da5adb26f3 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:26.904382 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f0ffb-89f6-46ec-a180-4f0172b52528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:26.905184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f0ffb-89f6-46ec-a180-4f0172b52528 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:26.905465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f0ffb-89f6-46ec-a180-4f0172b52528 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:26.905762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f0ffb-89f6-46ec-a180-4f0172b52528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:26.906298 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043914-b077-4bf2-8389-5bd8862c74e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:26.907072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043914-b077-4bf2-8389-5bd8862c74e0 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:26.907361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043914-b077-4bf2-8389-5bd8862c74e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:26.907642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043914-b077-4bf2-8389-5bd8862c74e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:26.915758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1043914-b077-4bf2-8389-5bd8862c74e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:26.916145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f0ffb-89f6-46ec-a180-4f0172b52528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:26.917215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1043914-b077-4bf2-8389-5bd8862c74e0 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:26.917559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f0ffb-89f6-46ec-a180-4f0172b52528 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-b1043914-b077-4bf2-8389-5bd8862c74e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:26.932397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:26.933409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:26.934414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:26.934414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:26.934414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:26.934414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:26.934414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:26.934414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:26.934414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:26.934414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:26.934414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:26.934414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:26.934414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 325/641] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 12:46:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:26.935023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043914-b077-4bf2-8389-5bd8862c74e0 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-ca3f0ffb-89f6-46ec-a180-4f0172b52528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:26.940031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:26.941421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:26.942481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:26.942481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:26.942481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:26.942481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:26.942481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 317/642] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:26.943374 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298cc8a-eaa9-45ba-97c9-96974d551300 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:26.943721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298cc8a-eaa9-45ba-97c9-96974d551300 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:26.944001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298cc8a-eaa9-45ba-97c9-96974d551300 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:26.949633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 318/643] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:26.950437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298cc8a-eaa9-45ba-97c9-96974d551300 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:27.018450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:27.043289 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043914-b077-4bf2-8389-5bd8862c74e0 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:27.063125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298cc8a-eaa9-45ba-97c9-96974d551300 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:27.078515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:27.078782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee64094-6794-4f67-9c6d-df01a0de3130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:27.079443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee64094-6794-4f67-9c6d-df01a0de3130 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:27.079728 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee64094-6794-4f67-9c6d-df01a0de3130 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:27.080181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee64094-6794-4f67-9c6d-df01a0de3130 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:27.080761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee64094-6794-4f67-9c6d-df01a0de3130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:27.081101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee64094-6794-4f67-9c6d-df01a0de3130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:27.082605 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee64094-6794-4f67-9c6d-df01a0de3130 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:27.098318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 326/644] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:27.098318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee64094-6794-4f67-9c6d-df01a0de3130 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:27.108148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e01027-fb5e-4853-bd82-30d2d8433b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:27.108768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e01027-fb5e-4853-bd82-30d2d8433b02 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:27.109569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e01027-fb5e-4853-bd82-30d2d8433b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:27.109841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e01027-fb5e-4853-bd82-30d2d8433b02 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:27.110699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e01027-fb5e-4853-bd82-30d2d8433b02 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:27.111007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e01027-fb5e-4853-bd82-30d2d8433b02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:27.112423 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e01027-fb5e-4853-bd82-30d2d8433b02 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:27.134640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 319/645] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 12:46:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:27.136637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c0c9-0f5e-437c-acad-62ebdb56545c None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:27.136935 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c0c9-0f5e-437c-acad-62ebdb56545c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:27.137276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c0c9-0f5e-437c-acad-62ebdb56545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:27.155319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e719c0c9-0f5e-437c-acad-62ebdb56545c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:27.157187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e719c0c9-0f5e-437c-acad-62ebdb56545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:27.198999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee64094-6794-4f67-9c6d-df01a0de3130 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:27.225556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 320/646] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:27.226420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719c0c9-0f5e-437c-acad-62ebdb56545c None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:27.240068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d1d59-1d90-48be-9246-9bf604b1751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:27.241080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d1d59-1d90-48be-9246-9bf604b1751b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:27.241557 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d1d59-1d90-48be-9246-9bf604b1751b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:27.242012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d1d59-1d90-48be-9246-9bf604b1751b None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:27.243046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d1d59-1d90-48be-9246-9bf604b1751b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:27.243546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d1d59-1d90-48be-9246-9bf604b1751b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:27.245209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d1d59-1d90-48be-9246-9bf604b1751b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:27.248564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:27.259326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 327/647] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:27.260357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d1d59-1d90-48be-9246-9bf604b1751b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:27.338568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:27.340954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719c0c9-0f5e-437c-acad-62ebdb56545c None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:27.369430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d1d59-1d90-48be-9246-9bf604b1751b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:27.377610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e499f5b-0195-4187-a164-10b638b7ee60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:27.378449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e499f5b-0195-4187-a164-10b638b7ee60 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:27.378864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e499f5b-0195-4187-a164-10b638b7ee60 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:27.379147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e499f5b-0195-4187-a164-10b638b7ee60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:27.386824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e499f5b-0195-4187-a164-10b638b7ee60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:27.388347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e499f5b-0195-4187-a164-10b638b7ee60 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-8e499f5b-0195-4187-a164-10b638b7ee60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:27.404050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:27.404973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:27.405911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:27.405911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:27.405911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:27.405911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:27.405911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:27.405911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:27.405911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:27.405911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:27.410205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 321/648] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:27.410903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e499f5b-0195-4187-a164-10b638b7ee60 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:27.412642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f60fa-58a8-471c-87c9-818b36cafcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:27.413373 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f60fa-58a8-471c-87c9-818b36cafcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:27.413669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f60fa-58a8-471c-87c9-818b36cafcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:27.413952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f60fa-58a8-471c-87c9-818b36cafcf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:27.478255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:27.485782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f60fa-58a8-471c-87c9-818b36cafcf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:27.487365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f60fa-58a8-471c-87c9-818b36cafcf0 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:27.522752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e499f5b-0195-4187-a164-10b638b7ee60 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:27.558601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25650b7-b6ce-44f2-abb6-cee6325f52e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:27.559470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25650b7-b6ce-44f2-abb6-cee6325f52e5 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:27.559751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25650b7-b6ce-44f2-abb6-cee6325f52e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:27.560021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25650b7-b6ce-44f2-abb6-cee6325f52e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:27.571774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25650b7-b6ce-44f2-abb6-cee6325f52e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-f25650b7-b6ce-44f2-abb6-cee6325f52e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:27.579504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:27.580461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 322/649] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:27.582585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25650b7-b6ce-44f2-abb6-cee6325f52e5 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:27.604671 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 328/650] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:27.604671 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f60fa-58a8-471c-87c9-818b36cafcf0 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:27.609462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:27.698492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25650b7-b6ce-44f2-abb6-cee6325f52e5 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:27.701813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:27.708579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f60fa-58a8-471c-87c9-818b36cafcf0 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:27.737376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc190a-11a8-40d8-a1b5-4bdf0a309e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:27.738202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc190a-11a8-40d8-a1b5-4bdf0a309e88 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:27.738515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc190a-11a8-40d8-a1b5-4bdf0a309e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:27.738792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc190a-11a8-40d8-a1b5-4bdf0a309e88 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:27.747447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc190a-11a8-40d8-a1b5-4bdf0a309e88 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:27.748910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc190a-11a8-40d8-a1b5-4bdf0a309e88 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:27.749019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03c5ab-2c48-4e3a-b207-72cb99dfa23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:27.750248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03c5ab-2c48-4e3a-b207-72cb99dfa23d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:27.750567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03c5ab-2c48-4e3a-b207-72cb99dfa23d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:27.750848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03c5ab-2c48-4e3a-b207-72cb99dfa23d None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:27.751655 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03c5ab-2c48-4e3a-b207-72cb99dfa23d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:27.751969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03c5ab-2c48-4e3a-b207-72cb99dfa23d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:27.753446 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03c5ab-2c48-4e3a-b207-72cb99dfa23d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-74bc190a-11a8-40d8-a1b5-4bdf0a309e88 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:27.765325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:27.766370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 323/651] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:27.772837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc190a-11a8-40d8-a1b5-4bdf0a309e88 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:27.773502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 329/652] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:27.773502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03c5ab-2c48-4e3a-b207-72cb99dfa23d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:27.855627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:27.876147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc190a-11a8-40d8-a1b5-4bdf0a309e88 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:27.876966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03c5ab-2c48-4e3a-b207-72cb99dfa23d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:27.912063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590be27b-2ae6-4dc7-99bf-528e77d9feb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:27.912792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590be27b-2ae6-4dc7-99bf-528e77d9feb6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:27.913069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590be27b-2ae6-4dc7-99bf-528e77d9feb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:27.913368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590be27b-2ae6-4dc7-99bf-528e77d9feb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:27.914083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590be27b-2ae6-4dc7-99bf-528e77d9feb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:27.914419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590be27b-2ae6-4dc7-99bf-528e77d9feb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:27.914529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a699ae-651b-43b6-b8fe-1f460e17a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:27.915378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a699ae-651b-43b6-b8fe-1f460e17a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:27.915670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a699ae-651b-43b6-b8fe-1f460e17a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:27.915843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590be27b-2ae6-4dc7-99bf-528e77d9feb6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:27.915946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a699ae-651b-43b6-b8fe-1f460e17a5c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:27.923930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 324/653] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:27.925029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590be27b-2ae6-4dc7-99bf-528e77d9feb6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:27.929754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a699ae-651b-43b6-b8fe-1f460e17a5c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:27.931241 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a699ae-651b-43b6-b8fe-1f460e17a5c4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-04a699ae-651b-43b6-b8fe-1f460e17a5c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:27.948876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:27.949767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:27.950707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:27.950707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:27.950707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:27.950707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:27.950707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 330/654] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:27.950990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a699ae-651b-43b6-b8fe-1f460e17a5c4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:27.961934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:28.023875 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590be27b-2ae6-4dc7-99bf-528e77d9feb6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:28.050421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a699ae-651b-43b6-b8fe-1f460e17a5c4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:28.055132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dced94-e243-43cb-b1cd-a07fd83d2c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:28.055798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dced94-e243-43cb-b1cd-a07fd83d2c6c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.056082 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dced94-e243-43cb-b1cd-a07fd83d2c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.056356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dced94-e243-43cb-b1cd-a07fd83d2c6c None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.057178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dced94-e243-43cb-b1cd-a07fd83d2c6c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:28.057481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dced94-e243-43cb-b1cd-a07fd83d2c6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:28.058927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dced94-e243-43cb-b1cd-a07fd83d2c6c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:28.072622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:28.082332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 325/655] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 12:46:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:28.082332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12929d-d1af-4a0f-8992-6e2c5d4b0b0f None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.082332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12929d-d1af-4a0f-8992-6e2c5d4b0b0f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.082332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12929d-d1af-4a0f-8992-6e2c5d4b0b0f None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.082332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 326/656] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:28.083057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12929d-d1af-4a0f-8992-6e2c5d4b0b0f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:28.083440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b63a72-175f-4aad-b56f-a8ed806e2ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:28.084076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b63a72-175f-4aad-b56f-a8ed806e2ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.084608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b63a72-175f-4aad-b56f-a8ed806e2ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.084881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b63a72-175f-4aad-b56f-a8ed806e2ad2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.085830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b63a72-175f-4aad-b56f-a8ed806e2ad2 None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=e47f1d65687e4cfcbce764076141702c) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:28.086146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b63a72-175f-4aad-b56f-a8ed806e2ad2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:28.087612 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b63a72-175f-4aad-b56f-a8ed806e2ad2 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:28.093992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 331/657] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:27 2022] GET /identity/v3/role_assignments?user.id=e47f1d65687e4cfcbce764076141702c&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:28.094784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b63a72-175f-4aad-b56f-a8ed806e2ad2 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:28.175391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:28.203384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12929d-d1af-4a0f-8992-6e2c5d4b0b0f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:28.208361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b63a72-175f-4aad-b56f-a8ed806e2ad2 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:28.243781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219406b4-57dc-44c9-a1cf-f3b102208c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:28.245071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219406b4-57dc-44c9-a1cf-f3b102208c27 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.245543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219406b4-57dc-44c9-a1cf-f3b102208c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.246024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4d275-670a-452a-97b4-cb2d38446a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:28.246187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219406b4-57dc-44c9-a1cf-f3b102208c27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.246725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4d275-670a-452a-97b4-cb2d38446a2a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.247037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4d275-670a-452a-97b4-cb2d38446a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.247313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4d275-670a-452a-97b4-cb2d38446a2a None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.248046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4d275-670a-452a-97b4-cb2d38446a2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:28.248352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4d275-670a-452a-97b4-cb2d38446a2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:28.249784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4d275-670a-452a-97b4-cb2d38446a2a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:28.258995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219406b4-57dc-44c9-a1cf-f3b102208c27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:28.261007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219406b4-57dc-44c9-a1cf-f3b102208c27 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:28.262986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 332/658] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:28.263837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4d275-670a-452a-97b4-cb2d38446a2a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-219406b4-57dc-44c9-a1cf-f3b102208c27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:28.287057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:28.288210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:28.289332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:28.289332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:28.289332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:28.289332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:28.290915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 327/659] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:28.292424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef878baf-562c-4608-84ef-ac670ffe5ea1 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.292717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef878baf-562c-4608-84ef-ac670ffe5ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.292994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef878baf-562c-4608-84ef-ac670ffe5ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.313407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-ef878baf-562c-4608-84ef-ac670ffe5ea1 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:28.314626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ef878baf-562c-4608-84ef-ac670ffe5ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:28.364411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:28.389057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4d275-670a-452a-97b4-cb2d38446a2a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:28.398343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 328/660] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:28.398343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef878baf-562c-4608-84ef-ac670ffe5ea1 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:28.425994 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:28.432328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95465d-9fbb-49f0-a212-e5e99a8c68ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:28.433219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95465d-9fbb-49f0-a212-e5e99a8c68ab None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.433341 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95465d-9fbb-49f0-a212-e5e99a8c68ab None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.433540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95465d-9fbb-49f0-a212-e5e99a8c68ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.507003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95465d-9fbb-49f0-a212-e5e99a8c68ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:28.507431 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef878baf-562c-4608-84ef-ac670ffe5ea1 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:28.508475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95465d-9fbb-49f0-a212-e5e99a8c68ab None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:28.543641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987d5e3-7b2a-4f0d-a0ba-b364587c1efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:28.544854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987d5e3-7b2a-4f0d-a0ba-b364587c1efb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.545146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987d5e3-7b2a-4f0d-a0ba-b364587c1efb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.545428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987d5e3-7b2a-4f0d-a0ba-b364587c1efb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.553079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987d5e3-7b2a-4f0d-a0ba-b364587c1efb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:28.554565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987d5e3-7b2a-4f0d-a0ba-b364587c1efb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-3987d5e3-7b2a-4f0d-a0ba-b364587c1efb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:28.569165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:28.570038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:28.570977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:28.570977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:28.570977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:28.570977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:28.570977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:28.570977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:28.570977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:28.570977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:28.570977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 329/661] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:28.572087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987d5e3-7b2a-4f0d-a0ba-b364587c1efb None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:28.617894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:28.624017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 333/662] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:28.624017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95465d-9fbb-49f0-a212-e5e99a8c68ab None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:28.658436 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:28.687449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987d5e3-7b2a-4f0d-a0ba-b364587c1efb None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:28.723381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a1649-378f-4c0c-bbdc-d06f7e267202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:28.724168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a1649-378f-4c0c-bbdc-d06f7e267202 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.724455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a1649-378f-4c0c-bbdc-d06f7e267202 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.724734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a1649-378f-4c0c-bbdc-d06f7e267202 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.725461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a1649-378f-4c0c-bbdc-d06f7e267202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:28.725770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a1649-378f-4c0c-bbdc-d06f7e267202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:28.727547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a1649-378f-4c0c-bbdc-d06f7e267202 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:28.735682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95465d-9fbb-49f0-a212-e5e99a8c68ab None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:28.742588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 330/663] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:28.743693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a1649-378f-4c0c-bbdc-d06f7e267202 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:28.773112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922e143-489e-4f90-80a0-a67778c65d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:28.774071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922e143-489e-4f90-80a0-a67778c65d93 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.774390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922e143-489e-4f90-80a0-a67778c65d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.774989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922e143-489e-4f90-80a0-a67778c65d93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.786900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922e143-489e-4f90-80a0-a67778c65d93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-a922e143-489e-4f90-80a0-a67778c65d93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:28.794718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:28.795724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:28.796651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 334/664] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:28.797345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922e143-489e-4f90-80a0-a67778c65d93 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:28.858092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a1649-378f-4c0c-bbdc-d06f7e267202 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:28.860551 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:28.896597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55915d3-e461-4c57-83c1-1f2b51ed9ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:28.897655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55915d3-e461-4c57-83c1-1f2b51ed9ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.898166 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55915d3-e461-4c57-83c1-1f2b51ed9ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.898625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55915d3-e461-4c57-83c1-1f2b51ed9ca6 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.899585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55915d3-e461-4c57-83c1-1f2b51ed9ca6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:28.900043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55915d3-e461-4c57-83c1-1f2b51ed9ca6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:28.901753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55915d3-e461-4c57-83c1-1f2b51ed9ca6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:28.904016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:28.914705 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922e143-489e-4f90-80a0-a67778c65d93 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:28.920258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 331/665] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:28.920258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55915d3-e461-4c57-83c1-1f2b51ed9ca6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:28.954946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce194d7-d57b-4234-b9d1-ff17371ccfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:28.955668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce194d7-d57b-4234-b9d1-ff17371ccfee None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:28.955952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce194d7-d57b-4234-b9d1-ff17371ccfee None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:28.956230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce194d7-d57b-4234-b9d1-ff17371ccfee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:28.966854 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce194d7-d57b-4234-b9d1-ff17371ccfee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:28.968863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce194d7-d57b-4234-b9d1-ff17371ccfee None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-5ce194d7-d57b-4234-b9d1-ff17371ccfee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:28.985887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:28.986870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:28.987766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:28.987766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:28.987766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:28.987766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:28.987766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 335/666] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:28.988417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce194d7-d57b-4234-b9d1-ff17371ccfee None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:29.030517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55915d3-e461-4c57-83c1-1f2b51ed9ca6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:29.066826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62c3ce-4652-4aa0-8cac-7bca1a9633ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:29.067632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62c3ce-4652-4aa0-8cac-7bca1a9633ab None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:29.067917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62c3ce-4652-4aa0-8cac-7bca1a9633ab None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:29.068197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62c3ce-4652-4aa0-8cac-7bca1a9633ab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:29.075883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62c3ce-4652-4aa0-8cac-7bca1a9633ab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:29.077692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62c3ce-4652-4aa0-8cac-7bca1a9633ab None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-9c62c3ce-4652-4aa0-8cac-7bca1a9633ab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:29.091636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:29.092579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 332/667] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:29.093579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62c3ce-4652-4aa0-8cac-7bca1a9633ab None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:29.104275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce194d7-d57b-4234-b9d1-ff17371ccfee None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:29.121237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:29.142918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b8540-c57d-439f-9e3c-bb973a3c7153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:29.143594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8540-c57d-439f-9e3c-bb973a3c7153 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:29.143902 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8540-c57d-439f-9e3c-bb973a3c7153 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:29.144162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8540-c57d-439f-9e3c-bb973a3c7153 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:29.144916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b8540-c57d-439f-9e3c-bb973a3c7153 None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:29.145230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b8540-c57d-439f-9e3c-bb973a3c7153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:29.146639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b8540-c57d-439f-9e3c-bb973a3c7153 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:29.159089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 336/668] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:29.159089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b8540-c57d-439f-9e3c-bb973a3c7153 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:29.167022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:29.207176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62c3ce-4652-4aa0-8cac-7bca1a9633ab None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:29.241046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ce132-63bf-4011-9e54-e6c97c802dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:29.241706 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce132-63bf-4011-9e54-e6c97c802dbc None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:29.242266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce132-63bf-4011-9e54-e6c97c802dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:29.242266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce132-63bf-4011-9e54-e6c97c802dbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:29.243124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ce132-63bf-4011-9e54-e6c97c802dbc None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=05b2edb6fcb7496bb55650080a9da172) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:29.243440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ce132-63bf-4011-9e54-e6c97c802dbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:29.244880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ce132-63bf-4011-9e54-e6c97c802dbc None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:29.251474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 333/669] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:29 2022] GET /identity/v3/role_assignments?user.id=05b2edb6fcb7496bb55650080a9da172&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:29.252602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ce132-63bf-4011-9e54-e6c97c802dbc None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:29.268767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b8540-c57d-439f-9e3c-bb973a3c7153 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:29.306036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ec9de-04ed-4a63-a172-82165c8b7e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:29.306748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ec9de-04ed-4a63-a172-82165c8b7e68 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:29.307038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ec9de-04ed-4a63-a172-82165c8b7e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:29.307318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ec9de-04ed-4a63-a172-82165c8b7e68 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:29.308060 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ec9de-04ed-4a63-a172-82165c8b7e68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:29.308369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ec9de-04ed-4a63-a172-82165c8b7e68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:29.309915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ec9de-04ed-4a63-a172-82165c8b7e68 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:29.328502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 337/670] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:29.328502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ec9de-04ed-4a63-a172-82165c8b7e68 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:29.371597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:29.400235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ce132-63bf-4011-9e54-e6c97c802dbc None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:29.409145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:29.444838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b329282-a19f-4fff-ba3f-7e21b00fa88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:29.445696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b329282-a19f-4fff-ba3f-7e21b00fa88a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:29.445976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b329282-a19f-4fff-ba3f-7e21b00fa88a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:29.446265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b329282-a19f-4fff-ba3f-7e21b00fa88a None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:29.447413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b329282-a19f-4fff-ba3f-7e21b00fa88a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:29.447621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b329282-a19f-4fff-ba3f-7e21b00fa88a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:29.449154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b329282-a19f-4fff-ba3f-7e21b00fa88a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:29.454116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ec9de-04ed-4a63-a172-82165c8b7e68 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:29.469853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 334/671] 10.222.0.45 () {56 vars in 1139 bytes} [Tue May 24 12:46:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:29.470878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b329282-a19f-4fff-ba3f-7e21b00fa88a None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:29.493493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7704b9-9f6a-471f-8735-4bdf90f503c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:29.494653 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7704b9-9f6a-471f-8735-4bdf90f503c6 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:29.495162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7704b9-9f6a-471f-8735-4bdf90f503c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:29.495654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7704b9-9f6a-471f-8735-4bdf90f503c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:29.508007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7704b9-9f6a-471f-8735-4bdf90f503c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:29.509794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7704b9-9f6a-471f-8735-4bdf90f503c6 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-1f7704b9-9f6a-471f-8735-4bdf90f503c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:29.529620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:29.530626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:29.531537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:29.531537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:29.531537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:29.531537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:29.538257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 338/672] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:29.539453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7704b9-9f6a-471f-8735-4bdf90f503c6 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:29.583896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b329282-a19f-4fff-ba3f-7e21b00fa88a None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:29.623696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74526b-811b-47ab-87ec-f7ddb30836ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:29.625156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74526b-811b-47ab-87ec-f7ddb30836ad None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:29.625449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74526b-811b-47ab-87ec-f7ddb30836ad None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:29.625725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74526b-811b-47ab-87ec-f7ddb30836ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:29.633172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74526b-811b-47ab-87ec-f7ddb30836ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:29.634828 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74526b-811b-47ab-87ec-f7ddb30836ad None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-0f74526b-811b-47ab-87ec-f7ddb30836ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:29.650549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:29.651460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:29.652366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:29.652366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:29.652366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:29.652366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:29.652366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:29.652366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:29.652366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:29.652366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:29.652366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 335/673] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:29.653439 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74526b-811b-47ab-87ec-f7ddb30836ad None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:29.657483 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7704b9-9f6a-471f-8735-4bdf90f503c6 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:29.657637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:29.680096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:29.703136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d42ed-fc6e-4890-aaa2-c004a7cf47ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:29.704222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d42ed-fc6e-4890-aaa2-c004a7cf47ec None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:29.704790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d42ed-fc6e-4890-aaa2-c004a7cf47ec None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:29.705078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d42ed-fc6e-4890-aaa2-c004a7cf47ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:29.766131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74526b-811b-47ab-87ec-f7ddb30836ad None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:29.804131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520274-cda2-4fe4-ad9a-74448ad84b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:29.804812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520274-cda2-4fe4-ad9a-74448ad84b5a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:29.805052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d42ed-fc6e-4890-aaa2-c004a7cf47ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:29.805291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520274-cda2-4fe4-ad9a-74448ad84b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:29.805514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520274-cda2-4fe4-ad9a-74448ad84b5a None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:29.806092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01520274-cda2-4fe4-ad9a-74448ad84b5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:29.806456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01520274-cda2-4fe4-ad9a-74448ad84b5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:29.806677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d42ed-fc6e-4890-aaa2-c004a7cf47ec None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:29.807824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01520274-cda2-4fe4-ad9a-74448ad84b5a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:29.822950 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 336/674] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:29.823792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520274-cda2-4fe4-ad9a-74448ad84b5a None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:29.894454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:29.936814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520274-cda2-4fe4-ad9a-74448ad84b5a None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:29.942506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 339/675] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:29.943455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d42ed-fc6e-4890-aaa2-c004a7cf47ec None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:29.966060 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:29.987227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c61d8d-d63b-43d6-a0ac-3083317c1536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:29.987913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c61d8d-d63b-43d6-a0ac-3083317c1536 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:29.988199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c61d8d-d63b-43d6-a0ac-3083317c1536 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:29.988481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c61d8d-d63b-43d6-a0ac-3083317c1536 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:29.989272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c61d8d-d63b-43d6-a0ac-3083317c1536 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:29.989585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c61d8d-d63b-43d6-a0ac-3083317c1536 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:29.991070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c61d8d-d63b-43d6-a0ac-3083317c1536 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:30.004149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 337/676] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:30.005150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c61d8d-d63b-43d6-a0ac-3083317c1536 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:30.066897 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d42ed-fc6e-4890-aaa2-c004a7cf47ec None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:30.108204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead0e2a-0c0f-466c-b36a-aadd9b2cd589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:30.109969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead0e2a-0c0f-466c-b36a-aadd9b2cd589 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:30.110861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead0e2a-0c0f-466c-b36a-aadd9b2cd589 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:30.111249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead0e2a-0c0f-466c-b36a-aadd9b2cd589 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:30.121832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead0e2a-0c0f-466c-b36a-aadd9b2cd589 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:30.123329 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead0e2a-0c0f-466c-b36a-aadd9b2cd589 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:30.132241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c61d8d-d63b-43d6-a0ac-3083317c1536 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-2ead0e2a-0c0f-466c-b36a-aadd9b2cd589 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:30.139529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:30.141765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 340/677] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:30.144224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead0e2a-0c0f-466c-b36a-aadd9b2cd589 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:30.174347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:30.176914 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdff20-0d8b-449c-92ad-e273ec42d803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:30.178754 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdff20-0d8b-449c-92ad-e273ec42d803 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:30.179271 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdff20-0d8b-449c-92ad-e273ec42d803 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:30.179659 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdff20-0d8b-449c-92ad-e273ec42d803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:30.202245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdff20-0d8b-449c-92ad-e273ec42d803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:30.203861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdff20-0d8b-449c-92ad-e273ec42d803 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-d4bdff20-0d8b-449c-92ad-e273ec42d803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:30.228040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:30.228999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:30.229900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:30.229900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:30.229900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:30.229900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:30.229900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 338/678] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:30.230650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdff20-0d8b-449c-92ad-e273ec42d803 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:30.279377 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:30.310205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead0e2a-0c0f-466c-b36a-aadd9b2cd589 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:30.346368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996be519-39db-47d9-b3e7-8ff49432eafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:30.347362 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996be519-39db-47d9-b3e7-8ff49432eafd None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:30.347865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996be519-39db-47d9-b3e7-8ff49432eafd None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:30.348336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996be519-39db-47d9-b3e7-8ff49432eafd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:30.349359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996be519-39db-47d9-b3e7-8ff49432eafd None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=8c9f08515d514be8af1176717ec9e6fd) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:30.349867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996be519-39db-47d9-b3e7-8ff49432eafd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:30.351611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996be519-39db-47d9-b3e7-8ff49432eafd None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:30.353563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdff20-0d8b-449c-92ad-e273ec42d803 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:30.362542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 341/679] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:30 2022] GET /identity/v3/role_assignments?user.id=8c9f08515d514be8af1176717ec9e6fd&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:30.363768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996be519-39db-47d9-b3e7-8ff49432eafd None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:30.398793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e928fd6-81ea-411c-835b-e40a9cebae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:30.399495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e928fd6-81ea-411c-835b-e40a9cebae43 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:30.399781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e928fd6-81ea-411c-835b-e40a9cebae43 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:30.400060 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e928fd6-81ea-411c-835b-e40a9cebae43 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:30.401519 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e928fd6-81ea-411c-835b-e40a9cebae43 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:30.401851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e928fd6-81ea-411c-835b-e40a9cebae43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:30.407496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e928fd6-81ea-411c-835b-e40a9cebae43 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:30.431933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 339/680] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 12:46:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:30.433388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae45be1-5ac7-468f-a376-11f96f45cb33 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:30.434236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae45be1-5ac7-468f-a376-11f96f45cb33 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:30.434554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae45be1-5ac7-468f-a376-11f96f45cb33 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:30.436087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 340/681] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:30.438614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae45be1-5ac7-468f-a376-11f96f45cb33 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:30.479480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:30.520723 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996be519-39db-47d9-b3e7-8ff49432eafd None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:30.554180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae45be1-5ac7-468f-a376-11f96f45cb33 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:30.560144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd853976-bb02-4a79-94b5-1250a09b7ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:30.560828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd853976-bb02-4a79-94b5-1250a09b7ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:30.561112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd853976-bb02-4a79-94b5-1250a09b7ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:30.561392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd853976-bb02-4a79-94b5-1250a09b7ca6 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:30.562689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd853976-bb02-4a79-94b5-1250a09b7ca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:30.563031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd853976-bb02-4a79-94b5-1250a09b7ca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:30.564516 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd853976-bb02-4a79-94b5-1250a09b7ca6 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:30.579801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 342/682] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:30.581204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8bf8b-0bff-4114-b0a4-482a66367a19 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:30.581497 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8bf8b-0bff-4114-b0a4-482a66367a19 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:30.581771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8bf8b-0bff-4114-b0a4-482a66367a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:30.587454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:30.596448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31e68d-4496-42cd-a137-07dd873925eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:30.597044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31e68d-4496-42cd-a137-07dd873925eb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:30.597332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31e68d-4496-42cd-a137-07dd873925eb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:30.597612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31e68d-4496-42cd-a137-07dd873925eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:30.601936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-1ed8bf8b-0bff-4114-b0a4-482a66367a19 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:30.603182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1ed8bf8b-0bff-4114-b0a4-482a66367a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:30.680812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31e68d-4496-42cd-a137-07dd873925eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:30.682425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31e68d-4496-42cd-a137-07dd873925eb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:30.689472 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 343/683] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:30.690634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8bf8b-0bff-4114-b0a4-482a66367a19 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:30.737516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:30.821731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 341/684] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:30.822724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8bf8b-0bff-4114-b0a4-482a66367a19 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:30.823193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31e68d-4496-42cd-a137-07dd873925eb None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:30.864548 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:30.874423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f7a5d-0161-4781-b0fe-be941b9d0253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:30.875772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f7a5d-0161-4781-b0fe-be941b9d0253 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:30.876259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f7a5d-0161-4781-b0fe-be941b9d0253 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:30.877218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f7a5d-0161-4781-b0fe-be941b9d0253 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:30.887711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f7a5d-0161-4781-b0fe-be941b9d0253 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:30.889970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f7a5d-0161-4781-b0fe-be941b9d0253 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-231f7a5d-0161-4781-b0fe-be941b9d0253 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:30.908475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:30.909560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:30.910697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:30.910697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:30.910697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:30.910697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:30.910697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:30.910697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:30.910697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:30.910697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:30.913471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 344/685] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:30.914907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f7a5d-0161-4781-b0fe-be941b9d0253 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:30.939471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31e68d-4496-42cd-a137-07dd873925eb None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:30.975258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eaf1ee-4b51-476e-adae-71b0cf0da3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:30.976221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaf1ee-4b51-476e-adae-71b0cf0da3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:30.976608 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaf1ee-4b51-476e-adae-71b0cf0da3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:30.977139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaf1ee-4b51-476e-adae-71b0cf0da3d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:30.979025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:30.989508 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eaf1ee-4b51-476e-adae-71b0cf0da3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:30.991148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eaf1ee-4b51-476e-adae-71b0cf0da3d0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-25eaf1ee-4b51-476e-adae-71b0cf0da3d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:31.009559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:31.010584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:31.011767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:31.011767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:31.011767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:31.011767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:31.016181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 342/686] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:31.017054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eaf1ee-4b51-476e-adae-71b0cf0da3d0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:31.050512 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f7a5d-0161-4781-b0fe-be941b9d0253 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:31.095489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50eab6-4f89-484b-8fec-030f9150c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:31.096319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50eab6-4f89-484b-8fec-030f9150c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:31.096603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50eab6-4f89-484b-8fec-030f9150c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:31.096882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50eab6-4f89-484b-8fec-030f9150c8d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:31.110815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50eab6-4f89-484b-8fec-030f9150c8d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:31.112286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50eab6-4f89-484b-8fec-030f9150c8d4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:31.128063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eaf1ee-4b51-476e-adae-71b0cf0da3d0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-ab50eab6-4f89-484b-8fec-030f9150c8d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:31.136715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:31.137611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:31.138534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:31.138534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:31.138534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:31.138534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:31.142720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 345/687] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:31.143561 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50eab6-4f89-484b-8fec-030f9150c8d4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:31.169929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fc1dd-4276-45ec-a7b3-97447e97e595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:31.170834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fc1dd-4276-45ec-a7b3-97447e97e595 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:31.171121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fc1dd-4276-45ec-a7b3-97447e97e595 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:31.171461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fc1dd-4276-45ec-a7b3-97447e97e595 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:31.172255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fc1dd-4276-45ec-a7b3-97447e97e595 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:31.172709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fc1dd-4276-45ec-a7b3-97447e97e595 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:31.174182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fc1dd-4276-45ec-a7b3-97447e97e595 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:31.187913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 343/688] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:31.188841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fc1dd-4276-45ec-a7b3-97447e97e595 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:31.205787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:31.241448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:31.281886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50eab6-4f89-484b-8fec-030f9150c8d4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:31.303860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fc1dd-4276-45ec-a7b3-97447e97e595 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:31.328101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0260963-fcfa-48fb-a8f0-8c5117726cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:31.329709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0260963-fcfa-48fb-a8f0-8c5117726cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:31.330308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0260963-fcfa-48fb-a8f0-8c5117726cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:31.330767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0260963-fcfa-48fb-a8f0-8c5117726cc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:31.332328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0260963-fcfa-48fb-a8f0-8c5117726cc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:31.332818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0260963-fcfa-48fb-a8f0-8c5117726cc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:31.334868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0260963-fcfa-48fb-a8f0-8c5117726cc5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:31.347650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403ab48-61e0-4953-99d7-c3ee5bba8494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:31.348656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ab48-61e0-4953-99d7-c3ee5bba8494 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:31.349153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ab48-61e0-4953-99d7-c3ee5bba8494 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:31.349625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ab48-61e0-4953-99d7-c3ee5bba8494 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:31.350654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403ab48-61e0-4953-99d7-c3ee5bba8494 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:31.351156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403ab48-61e0-4953-99d7-c3ee5bba8494 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:31.352717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403ab48-61e0-4953-99d7-c3ee5bba8494 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:31.353072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 346/689] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:31.354089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0260963-fcfa-48fb-a8f0-8c5117726cc5 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:31.373201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 344/690] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:31.373781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403ab48-61e0-4953-99d7-c3ee5bba8494 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:31.469223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0260963-fcfa-48fb-a8f0-8c5117726cc5 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:31.472496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:31.505298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403ab48-61e0-4953-99d7-c3ee5bba8494 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:31.514901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:31.518563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e7d5b-0936-4aef-950a-8c348fd5804e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:31.518960 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e7d5b-0936-4aef-950a-8c348fd5804e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:31.519261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e7d5b-0936-4aef-950a-8c348fd5804e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:31.519556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e7d5b-0936-4aef-950a-8c348fd5804e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:31.520677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e7d5b-0936-4aef-950a-8c348fd5804e None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=bb111be6bc2946d3b50b8d99a89cf8ac) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:31.521001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e7d5b-0936-4aef-950a-8c348fd5804e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:31.522544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e7d5b-0936-4aef-950a-8c348fd5804e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:31.530151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 347/691] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:31 2022] GET /identity/v3/role_assignments?user.id=bb111be6bc2946d3b50b8d99a89cf8ac&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:31.531209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e7d5b-0936-4aef-950a-8c348fd5804e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:31.564288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58ffd2-20e3-4746-9145-1d491e3ebe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:31.564991 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58ffd2-20e3-4746-9145-1d491e3ebe82 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:31.565275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58ffd2-20e3-4746-9145-1d491e3ebe82 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:31.565554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58ffd2-20e3-4746-9145-1d491e3ebe82 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:31.566305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58ffd2-20e3-4746-9145-1d491e3ebe82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:31.569807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58ffd2-20e3-4746-9145-1d491e3ebe82 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:31.615030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 345/692] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:46:31 2022] POST /identity/v3/users => generated 254 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:46:31.682466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e7d5b-0936-4aef-950a-8c348fd5804e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:31.716251 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9a28f-0969-4c51-9d0a-8f44a6b5a412 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:31.716584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9a28f-0969-4c51-9d0a-8f44a6b5a412 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:31.716819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9a28f-0969-4c51-9d0a-8f44a6b5a412 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:31.718801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 346/693] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:46:31.736584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4129b-e870-4a4f-b684-1c2c43e1bd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:31.737368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4129b-e870-4a4f-b684-1c2c43e1bd0c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:31.737654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4129b-e870-4a4f-b684-1c2c43e1bd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:31.737921 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4129b-e870-4a4f-b684-1c2c43e1bd0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:31.738474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2f86-d7ea-4825-85df-ef3d5b5c7a4c None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:31.738775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2f86-d7ea-4825-85df-ef3d5b5c7a4c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:31.739057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2f86-d7ea-4825-85df-ef3d5b5c7a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:31.760994 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4129b-e870-4a4f-b684-1c2c43e1bd0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:31.762535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4129b-e870-4a4f-b684-1c2c43e1bd0c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:31.768816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-884d2f86-d7ea-4825-85df-ef3d5b5c7a4c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:31.770087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-884d2f86-d7ea-4825-85df-ef3d5b5c7a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:31.869772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 347/694] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:31.899330 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d2f86-d7ea-4825-85df-ef3d5b5c7a4c None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-8cc4129b-e870-4a4f-b684-1c2c43e1bd0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:31.925061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:31.926019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:31.927558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:31.927558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:31.927558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:31.927558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:31.927558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:31.927558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:31.927558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:31.927558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:31.927558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 348/695] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:31.933607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4129b-e870-4a4f-b684-1c2c43e1bd0c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:31.974786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:32.008759 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:32.025349 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d2f86-d7ea-4825-85df-ef3d5b5c7a4c None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:32.066231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6f97a-47fe-4519-abf1-d65636be972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:32.067051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6f97a-47fe-4519-abf1-d65636be972e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:32.067338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6f97a-47fe-4519-abf1-d65636be972e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:32.067613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6f97a-47fe-4519-abf1-d65636be972e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:32.077450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4129b-e870-4a4f-b684-1c2c43e1bd0c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:32.081091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6f97a-47fe-4519-abf1-d65636be972e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-99b6f97a-47fe-4519-abf1-d65636be972e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:32.089003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:32.089963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:32.090848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:32.095880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 348/696] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:32.096770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6f97a-47fe-4519-abf1-d65636be972e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:32.124783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208a7e6-c5c4-46a9-b6c9-8eb1dbb12669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:32.126077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208a7e6-c5c4-46a9-b6c9-8eb1dbb12669 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:32.126908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208a7e6-c5c4-46a9-b6c9-8eb1dbb12669 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:32.127776 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208a7e6-c5c4-46a9-b6c9-8eb1dbb12669 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:32.129043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208a7e6-c5c4-46a9-b6c9-8eb1dbb12669 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:32.129571 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208a7e6-c5c4-46a9-b6c9-8eb1dbb12669 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:32.132180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208a7e6-c5c4-46a9-b6c9-8eb1dbb12669 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:32.149332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 349/697] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:32.155052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208a7e6-c5c4-46a9-b6c9-8eb1dbb12669 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:32.220385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6f97a-47fe-4519-abf1-d65636be972e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:32.258964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025084d-6908-45da-9325-76a8eb420f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:32.259773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025084d-6908-45da-9325-76a8eb420f6a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:32.260315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025084d-6908-45da-9325-76a8eb420f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:32.260596 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025084d-6908-45da-9325-76a8eb420f6a None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:32.261664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025084d-6908-45da-9325-76a8eb420f6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:32.261978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025084d-6908-45da-9325-76a8eb420f6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:32.263864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025084d-6908-45da-9325-76a8eb420f6a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:32.274922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 349/698] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:32.281265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025084d-6908-45da-9325-76a8eb420f6a None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:32.312960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:32.347166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:32.364761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208a7e6-c5c4-46a9-b6c9-8eb1dbb12669 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:32.418462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c99045-a78c-4e73-ae11-f89ca3b84c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:32.418462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99045-a78c-4e73-ae11-f89ca3b84c3a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:32.418462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99045-a78c-4e73-ae11-f89ca3b84c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:32.418462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99045-a78c-4e73-ae11-f89ca3b84c3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:32.418818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c99045-a78c-4e73-ae11-f89ca3b84c3a None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=d00b728319304b1e9ba2f1ee93022187) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:32.418818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c99045-a78c-4e73-ae11-f89ca3b84c3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:32.418818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c99045-a78c-4e73-ae11-f89ca3b84c3a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:32.439877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 350/699] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:32 2022] GET /identity/v3/role_assignments?user.id=d00b728319304b1e9ba2f1ee93022187&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:32.454629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025084d-6908-45da-9325-76a8eb420f6a None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:32.523240 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a009fa-d5b8-4137-b5fd-3dbc4d03e06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:32.523993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a009fa-d5b8-4137-b5fd-3dbc4d03e06d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:32.524281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a009fa-d5b8-4137-b5fd-3dbc4d03e06d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:32.524561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a009fa-d5b8-4137-b5fd-3dbc4d03e06d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:32.538030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a009fa-d5b8-4137-b5fd-3dbc4d03e06d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:32.539543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a009fa-d5b8-4137-b5fd-3dbc4d03e06d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-b4a009fa-d5b8-4137-b5fd-3dbc4d03e06d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:32.558147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:32.559138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:32.560037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:32.560037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:32.560037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:32.560037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:32.561579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 350/700] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:32.581107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c99045-a78c-4e73-ae11-f89ca3b84c3a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:32.703498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c99045-a78c-4e73-ae11-f89ca3b84c3a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:32.745917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1ca4f-ed68-4eb8-8cd3-c7d69560b504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:32.746638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1ca4f-ed68-4eb8-8cd3-c7d69560b504 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:32.746929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1ca4f-ed68-4eb8-8cd3-c7d69560b504 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:32.747211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1ca4f-ed68-4eb8-8cd3-c7d69560b504 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:32.747964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1ca4f-ed68-4eb8-8cd3-c7d69560b504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:32.748285 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1ca4f-ed68-4eb8-8cd3-c7d69560b504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:32.750250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1ca4f-ed68-4eb8-8cd3-c7d69560b504 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:32.767130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 351/701] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:32.772507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a009fa-d5b8-4137-b5fd-3dbc4d03e06d None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:32.855633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:32.896816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a009fa-d5b8-4137-b5fd-3dbc4d03e06d None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:32.932148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82282c-2679-4ef6-8b10-1347cd518717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:32.932953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82282c-2679-4ef6-8b10-1347cd518717 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:32.933300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82282c-2679-4ef6-8b10-1347cd518717 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:32.933579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82282c-2679-4ef6-8b10-1347cd518717 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:32.941722 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82282c-2679-4ef6-8b10-1347cd518717 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:32.943478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82282c-2679-4ef6-8b10-1347cd518717 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:32.949544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8707f-fcad-4b79-8022-4ef6a30932c3 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:32.949883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8707f-fcad-4b79-8022-4ef6a30932c3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:32.950165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8707f-fcad-4b79-8022-4ef6a30932c3 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:32.951949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 352/702] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:32.957756 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440710b1-1821-4b69-ade0-ab54431e0229 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:32.958041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440710b1-1821-4b69-ade0-ab54431e0229 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:32.958358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440710b1-1821-4b69-ade0-ab54431e0229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-9b82282c-2679-4ef6-8b10-1347cd518717 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:32.961544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:32.962500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:32.964131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:32.964131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:32.964131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:32.964131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:32.964131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:32.964131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:32.964131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:32.964131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:32.964131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:32.964131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:32.964131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 351/703] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 12:46:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:32.968643 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82282c-2679-4ef6-8b10-1347cd518717 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:32.978082 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-440710b1-1821-4b69-ade0-ab54431e0229 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:32.979296 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-440710b1-1821-4b69-ade0-ab54431e0229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:32.989247 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:33.075750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 353/704] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:33.085346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82282c-2679-4ef6-8b10-1347cd518717 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:33.100377 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440710b1-1821-4b69-ade0-ab54431e0229 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:33.115068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:33.125622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e7159-1b83-433f-b36a-a24868052342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:33.126457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e7159-1b83-433f-b36a-a24868052342 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:33.126762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e7159-1b83-433f-b36a-a24868052342 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:33.127023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e7159-1b83-433f-b36a-a24868052342 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:33.128004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e7159-1b83-433f-b36a-a24868052342 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:33.128321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e7159-1b83-433f-b36a-a24868052342 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:33.129880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e7159-1b83-433f-b36a-a24868052342 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:33.151085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 352/705] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 12:46:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:33.159008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e7159-1b83-433f-b36a-a24868052342 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:33.232108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440710b1-1821-4b69-ade0-ab54431e0229 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:33.271073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e7159-1b83-433f-b36a-a24868052342 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:33.283996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d967c4-53ca-4be7-8da2-82a1932c8b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:33.284815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d967c4-53ca-4be7-8da2-82a1932c8b6f None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:33.285109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d967c4-53ca-4be7-8da2-82a1932c8b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:33.285391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d967c4-53ca-4be7-8da2-82a1932c8b6f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:33.286097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d967c4-53ca-4be7-8da2-82a1932c8b6f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:33.287544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d967c4-53ca-4be7-8da2-82a1932c8b6f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-b3d967c4-53ca-4be7-8da2-82a1932c8b6f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:33.302807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:33.303853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:33.304698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 354/706] 10.222.0.45 () {56 vars in 1129 bytes} [Tue May 24 12:46:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:33.311206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d967c4-53ca-4be7-8da2-82a1932c8b6f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:33.328059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce977dbe-795b-4e58-9562-bde7a7c2c32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:33.328969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce977dbe-795b-4e58-9562-bde7a7c2c32a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:33.329400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce977dbe-795b-4e58-9562-bde7a7c2c32a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:33.329847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce977dbe-795b-4e58-9562-bde7a7c2c32a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:33.348103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:33.423407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:33.453026 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce977dbe-795b-4e58-9562-bde7a7c2c32a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:33.454593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce977dbe-795b-4e58-9562-bde7a7c2c32a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:33.479951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d967c4-53ca-4be7-8da2-82a1932c8b6f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:33.554995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969d83e-3858-4c68-9881-249ce4d5bed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:33.555680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969d83e-3858-4c68-9881-249ce4d5bed3 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:33.555972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969d83e-3858-4c68-9881-249ce4d5bed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:33.556251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969d83e-3858-4c68-9881-249ce4d5bed3 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:33.556974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969d83e-3858-4c68-9881-249ce4d5bed3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:33.557284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969d83e-3858-4c68-9881-249ce4d5bed3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:33.558701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969d83e-3858-4c68-9881-249ce4d5bed3 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:33.569162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 355/707] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 12:46:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:33.583290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969d83e-3858-4c68-9881-249ce4d5bed3 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:33.601347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 353/708] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:33.606760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce977dbe-795b-4e58-9562-bde7a7c2c32a None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:33.699679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:33.710979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce977dbe-795b-4e58-9562-bde7a7c2c32a None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:33.747172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac830d-8db9-4d77-a2bc-a990c5142d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:33.747895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:33.748034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac830d-8db9-4d77-a2bc-a990c5142d37 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:33.748254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac830d-8db9-4d77-a2bc-a990c5142d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:33.748533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac830d-8db9-4d77-a2bc-a990c5142d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:33.759436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac830d-8db9-4d77-a2bc-a990c5142d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:33.761157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac830d-8db9-4d77-a2bc-a990c5142d37 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-95ac830d-8db9-4d77-a2bc-a990c5142d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:33.779663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:33.780682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:33.781589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:33.781589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:33.781589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:33.781589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:33.781589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 354/709] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:33.786555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac830d-8db9-4d77-a2bc-a990c5142d37 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:33.833094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969d83e-3858-4c68-9881-249ce4d5bed3 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:33.890906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac830d-8db9-4d77-a2bc-a990c5142d37 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:33.923021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823857-85fa-45f2-b78c-56ec29607425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:33.923710 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823857-85fa-45f2-b78c-56ec29607425 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:33.923999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823857-85fa-45f2-b78c-56ec29607425 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:33.924280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823857-85fa-45f2-b78c-56ec29607425 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:33.925004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b823857-85fa-45f2-b78c-56ec29607425 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:33.925385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b823857-85fa-45f2-b78c-56ec29607425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:33.926724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b823857-85fa-45f2-b78c-56ec29607425 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:33.926909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e11612-0ceb-4e1c-bb1c-bc3c0b7f4826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:33.927547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e11612-0ceb-4e1c-bb1c-bc3c0b7f4826 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:33.927834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e11612-0ceb-4e1c-bb1c-bc3c0b7f4826 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:33.928226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e11612-0ceb-4e1c-bb1c-bc3c0b7f4826 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:33.928960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e11612-0ceb-4e1c-bb1c-bc3c0b7f4826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:33.929271 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e11612-0ceb-4e1c-bb1c-bc3c0b7f4826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:33.930905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e11612-0ceb-4e1c-bb1c-bc3c0b7f4826 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:33.938779 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 356/710] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 12:46:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:33.942653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:33.948040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 355/711] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:33.953469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823857-85fa-45f2-b78c-56ec29607425 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:34.074692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823857-85fa-45f2-b78c-56ec29607425 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:34.116611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e45b0c-5b9c-41a8-a062-4c21ab0463cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:34.117432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e45b0c-5b9c-41a8-a062-4c21ab0463cc None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:34.117720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e45b0c-5b9c-41a8-a062-4c21ab0463cc None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:34.118002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e45b0c-5b9c-41a8-a062-4c21ab0463cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:34.120195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:34.128231 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e45b0c-5b9c-41a8-a062-4c21ab0463cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:34.129719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e45b0c-5b9c-41a8-a062-4c21ab0463cc None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-78e45b0c-5b9c-41a8-a062-4c21ab0463cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:34.145587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:34.146642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:34.147582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:34.147582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:34.147582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:34.147582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:34.147582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:34.147582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:34.147582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:34.147582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:34.147582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 357/712] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:34.152689 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e11612-0ceb-4e1c-bb1c-bc3c0b7f4826 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:34.267988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fee0e-b88b-4a17-8e57-6fe0d81b7104 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:34.268378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fee0e-b88b-4a17-8e57-6fe0d81b7104 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:34.268762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fee0e-b88b-4a17-8e57-6fe0d81b7104 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:34.270422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 358/713] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:34.275505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be39c20-09cd-46cd-a176-382da808962e None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:34.275924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be39c20-09cd-46cd-a176-382da808962e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:34.276306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be39c20-09cd-46cd-a176-382da808962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:34.289433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e11612-0ceb-4e1c-bb1c-bc3c0b7f4826 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:34.319607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8be39c20-09cd-46cd-a176-382da808962e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:34.321007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8be39c20-09cd-46cd-a176-382da808962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:34.327069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594c413-8bef-49e6-ad4f-f79a856c055a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:34.327828 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594c413-8bef-49e6-ad4f-f79a856c055a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:34.328169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594c413-8bef-49e6-ad4f-f79a856c055a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:34.328445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594c413-8bef-49e6-ad4f-f79a856c055a None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:34.329303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594c413-8bef-49e6-ad4f-f79a856c055a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:34.329616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594c413-8bef-49e6-ad4f-f79a856c055a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:34.331224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594c413-8bef-49e6-ad4f-f79a856c055a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:34.345063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 356/714] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:34.350922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594c413-8bef-49e6-ad4f-f79a856c055a None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:34.414688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:34.465326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 359/715] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:34.473532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594c413-8bef-49e6-ad4f-f79a856c055a None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:34.500207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be39c20-09cd-46cd-a176-382da808962e None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:34.520299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a580e30-0580-4686-a99b-6d697c13248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:34.521286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a580e30-0580-4686-a99b-6d697c13248d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:34.521665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a580e30-0580-4686-a99b-6d697c13248d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:34.522148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a580e30-0580-4686-a99b-6d697c13248d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:34.523100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a580e30-0580-4686-a99b-6d697c13248d None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=05b2edb6fcb7496bb55650080a9da172) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:34.523509 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a580e30-0580-4686-a99b-6d697c13248d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:34.525033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a580e30-0580-4686-a99b-6d697c13248d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:34.531150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 357/716] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:34 2022] GET /identity/v3/role_assignments?user.id=05b2edb6fcb7496bb55650080a9da172&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=0c473678b281495f86f58b077be04e1b => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:34.538020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f697c9-1cb4-4a73-8bb7-07e7e2bed1b7 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:34.538020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f697c9-1cb4-4a73-8bb7-07e7e2bed1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:34.538020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f697c9-1cb4-4a73-8bb7-07e7e2bed1b7 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:34.538348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 358/717] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:34.539900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae0fe3-0820-4b7e-9a8b-00721bbd5d27 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:34.540241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae0fe3-0820-4b7e-9a8b-00721bbd5d27 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:34.540588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae0fe3-0820-4b7e-9a8b-00721bbd5d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:34.558437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9bae0fe3-0820-4b7e-9a8b-00721bbd5d27 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:34.562310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9bae0fe3-0820-4b7e-9a8b-00721bbd5d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:34.661558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be39c20-09cd-46cd-a176-382da808962e None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:34.687405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 359/718] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:34.699352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:34.711564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f883ba-9c93-4535-8048-90eebeae465e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:34.712618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f883ba-9c93-4535-8048-90eebeae465e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:34.713111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f883ba-9c93-4535-8048-90eebeae465e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:34.713560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f883ba-9c93-4535-8048-90eebeae465e None admin] PATH_INFO: `/v3/services/network` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:34.714414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f883ba-9c93-4535-8048-90eebeae465e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:34.715971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f883ba-9c93-4535-8048-90eebeae465e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:34.721207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae0fe3-0820-4b7e-9a8b-00721bbd5d27 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-e4f883ba-9c93-4535-8048-90eebeae465e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:34.753499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:34.754904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 12:46:34.755979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:34.760893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 360/719] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:34 2022] GET /identity/v3/services/network => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:34.764630 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f883ba-9c93-4535-8048-90eebeae465e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:34.816848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:34.875938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae0fe3-0820-4b7e-9a8b-00721bbd5d27 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:34.918746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef918a0f-c94b-4612-adbe-1a0677ccbe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:34.920703 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef918a0f-c94b-4612-adbe-1a0677ccbe4c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:34.921181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef918a0f-c94b-4612-adbe-1a0677ccbe4c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:34.921621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef918a0f-c94b-4612-adbe-1a0677ccbe4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:34.952516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef918a0f-c94b-4612-adbe-1a0677ccbe4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:34.954758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f883ba-9c93-4535-8048-90eebeae465e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-ef918a0f-c94b-4612-adbe-1a0677ccbe4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:34.960521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:34.961462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:34.962331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:34.970480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 360/720] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:34.976054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef918a0f-c94b-4612-adbe-1a0677ccbe4c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:35.007650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3f3fe-faef-4700-a5bc-844554121288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:35.008565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3f3fe-faef-4700-a5bc-844554121288 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:35.008808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3f3fe-faef-4700-a5bc-844554121288 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:35.009090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3f3fe-faef-4700-a5bc-844554121288 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:35.009813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3f3fe-faef-4700-a5bc-844554121288 None admin] RBAC: Adding query filter params (name=network) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:35.011251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3f3fe-faef-4700-a5bc-844554121288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:35.014784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3f3fe-faef-4700-a5bc-844554121288 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:35.024031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 361/721] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:46:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:35.031835 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3f3fe-faef-4700-a5bc-844554121288 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:35.118735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:35.169697 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef918a0f-c94b-4612-adbe-1a0677ccbe4c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:35.188008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:35.210873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3f3fe-faef-4700-a5bc-844554121288 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:35.244259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d710b0-96ab-4487-ad6f-a2985ebeb8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:35.244990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d710b0-96ab-4487-ad6f-a2985ebeb8ce None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:35.245273 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d710b0-96ab-4487-ad6f-a2985ebeb8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:35.245543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d710b0-96ab-4487-ad6f-a2985ebeb8ce None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:35.246665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d710b0-96ab-4487-ad6f-a2985ebeb8ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:35.246973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d710b0-96ab-4487-ad6f-a2985ebeb8ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:35.248347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d710b0-96ab-4487-ad6f-a2985ebeb8ce None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:35.264592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 361/722] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:35.273999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d710b0-96ab-4487-ad6f-a2985ebeb8ce None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:35.280507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054bd05-9030-4217-a3a2-df7818ca8eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:35.281189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054bd05-9030-4217-a3a2-df7818ca8eac None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:35.281471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054bd05-9030-4217-a3a2-df7818ca8eac None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:35.281744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054bd05-9030-4217-a3a2-df7818ca8eac None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:35.282480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054bd05-9030-4217-a3a2-df7818ca8eac None admin] RBAC: Adding query filter params (type=network) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:35.282787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054bd05-9030-4217-a3a2-df7818ca8eac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:35.284163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054bd05-9030-4217-a3a2-df7818ca8eac None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:35.302489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 362/723] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:46:35 2022] GET /identity/v3/services?type=network => generated 124 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:35.431624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d710b0-96ab-4487-ad6f-a2985ebeb8ce None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:35.472270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983643c0-477d-40e3-acf0-c93c07e1c067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:35.473104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983643c0-477d-40e3-acf0-c93c07e1c067 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:35.473389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983643c0-477d-40e3-acf0-c93c07e1c067 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:35.473663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983643c0-477d-40e3-acf0-c93c07e1c067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:35.491001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983643c0-477d-40e3-acf0-c93c07e1c067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:35.492467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983643c0-477d-40e3-acf0-c93c07e1c067 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-983643c0-477d-40e3-acf0-c93c07e1c067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:35.526132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:35.529068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:35.532826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:35.532826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:35.532826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:35.532826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:35.532826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 362/724] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:35.538315 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054bd05-9030-4217-a3a2-df7818ca8eac None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:35.702048 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054bd05-9030-4217-a3a2-df7818ca8eac None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:35.721898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:35.748115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f395d-082d-4cb9-a78c-d496d6850cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:35.748789 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f395d-082d-4cb9-a78c-d496d6850cad None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:35.749075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f395d-082d-4cb9-a78c-d496d6850cad None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:35.749886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f395d-082d-4cb9-a78c-d496d6850cad None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:35.750674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f395d-082d-4cb9-a78c-d496d6850cad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:35.750989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f395d-082d-4cb9-a78c-d496d6850cad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:35.752374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f395d-082d-4cb9-a78c-d496d6850cad None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:35.775667 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 363/725] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:35.781234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983643c0-477d-40e3-acf0-c93c07e1c067 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:35.818058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af023c7-933d-4ddb-a5bd-6ddda6ad144d None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:35.818445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af023c7-933d-4ddb-a5bd-6ddda6ad144d None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:35.818625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af023c7-933d-4ddb-a5bd-6ddda6ad144d None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:35.820108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 364/726] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:35.828101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e28233-a095-440a-9ffd-cc07296054e2 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:35.828332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:35.829332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e28233-a095-440a-9ffd-cc07296054e2 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:35.829332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e28233-a095-440a-9ffd-cc07296054e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:35.848404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-24e28233-a095-440a-9ffd-cc07296054e2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:35.853117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-24e28233-a095-440a-9ffd-cc07296054e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:35.982407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 365/727] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 12:46:36.001426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983643c0-477d-40e3-acf0-c93c07e1c067 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:36.036282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e28233-a095-440a-9ffd-cc07296054e2 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:36.037280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383a5f2-da70-467f-b0ed-f7db52134f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:36.038118 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383a5f2-da70-467f-b0ed-f7db52134f09 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:36.038476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383a5f2-da70-467f-b0ed-f7db52134f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:36.038698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383a5f2-da70-467f-b0ed-f7db52134f09 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:36.047815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383a5f2-da70-467f-b0ed-f7db52134f09 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:36.049269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383a5f2-da70-467f-b0ed-f7db52134f09 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-f383a5f2-da70-467f-b0ed-f7db52134f09 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:36.065149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:36.066124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:36.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:36.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:36.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:36.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:36.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:36.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:36.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:36.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:36.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:46:36.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:36.067105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 363/728] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 12:46:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:36.074055 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383a5f2-da70-467f-b0ed-f7db52134f09 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:36.224432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e28233-a095-440a-9ffd-cc07296054e2 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:36.235142 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:36.244471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:36.275683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383a5f2-da70-467f-b0ed-f7db52134f09 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:36.290424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c453be9-bfe2-4782-adde-f9917f868401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:36.291390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453be9-bfe2-4782-adde-f9917f868401 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:36.291706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453be9-bfe2-4782-adde-f9917f868401 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:36.291987 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453be9-bfe2-4782-adde-f9917f868401 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:36.303411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c453be9-bfe2-4782-adde-f9917f868401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-5c453be9-bfe2-4782-adde-f9917f868401 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:36.311213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:36.312712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:36.313562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 366/729] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:36.319545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c453be9-bfe2-4782-adde-f9917f868401 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:36.345284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525893aa-e81d-492c-9b03-f22f6c4218d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:36.346760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525893aa-e81d-492c-9b03-f22f6c4218d6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:36.347042 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525893aa-e81d-492c-9b03-f22f6c4218d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:36.347317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525893aa-e81d-492c-9b03-f22f6c4218d6 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:36.348150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525893aa-e81d-492c-9b03-f22f6c4218d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:36.348457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525893aa-e81d-492c-9b03-f22f6c4218d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:36.349890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525893aa-e81d-492c-9b03-f22f6c4218d6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:36.396718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 364/730] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 12:46:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:36.402823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525893aa-e81d-492c-9b03-f22f6c4218d6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:36.547465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c453be9-bfe2-4782-adde-f9917f868401 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:36.587447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525893aa-e81d-492c-9b03-f22f6c4218d6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:36.613645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78155fd8-a694-48e0-a7d7-f04c8675edfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:36.614267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78155fd8-a694-48e0-a7d7-f04c8675edfd None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:36.614574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78155fd8-a694-48e0-a7d7-f04c8675edfd None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:36.614851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78155fd8-a694-48e0-a7d7-f04c8675edfd None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:36.615609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78155fd8-a694-48e0-a7d7-f04c8675edfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:36.615914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78155fd8-a694-48e0-a7d7-f04c8675edfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:36.617299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78155fd8-a694-48e0-a7d7-f04c8675edfd None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:36.640520 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 367/731] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:36.642640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78155fd8-a694-48e0-a7d7-f04c8675edfd None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:36.653810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a2f38-d01d-4659-8203-c6c319c22407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:36.654890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a2f38-d01d-4659-8203-c6c319c22407 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:36.655471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a2f38-d01d-4659-8203-c6c319c22407 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:36.655986 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a2f38-d01d-4659-8203-c6c319c22407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:36.676076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:36.701938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:36.762910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a2f38-d01d-4659-8203-c6c319c22407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:36.764602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a2f38-d01d-4659-8203-c6c319c22407 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:36.786567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78155fd8-a694-48e0-a7d7-f04c8675edfd None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:36.830821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc17ca-69e7-45dd-8311-36a50506925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:36.832201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc17ca-69e7-45dd-8311-36a50506925b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:36.832650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc17ca-69e7-45dd-8311-36a50506925b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:36.833089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc17ca-69e7-45dd-8311-36a50506925b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:36.845905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc17ca-69e7-45dd-8311-36a50506925b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:36.847738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc17ca-69e7-45dd-8311-36a50506925b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-59dc17ca-69e7-45dd-8311-36a50506925b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:36.865065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:36.866004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:36.866922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:36.866922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:36.866922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:36.866922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:36.873358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 368/732] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:36.874988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563390c0-48b8-4f03-b34d-3f5a9444f17b None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:36.875563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563390c0-48b8-4f03-b34d-3f5a9444f17b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:36.876038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563390c0-48b8-4f03-b34d-3f5a9444f17b None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:36.881545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 369/733] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:36.882565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563390c0-48b8-4f03-b34d-3f5a9444f17b None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:36.897763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 365/734] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 12:46:36.897763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e387-2d00-4e23-b58d-2c0f9ac1d9e4 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:36.897763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e387-2d00-4e23-b58d-2c0f9ac1d9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:36.897763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e387-2d00-4e23-b58d-2c0f9ac1d9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:36.916988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7877e387-2d00-4e23-b58d-2c0f9ac1d9e4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:36.918185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7877e387-2d00-4e23-b58d-2c0f9ac1d9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:36.955201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:36.985021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:36.994313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 366/735] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:36.994313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877e387-2d00-4e23-b58d-2c0f9ac1d9e4 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:37.026075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563390c0-48b8-4f03-b34d-3f5a9444f17b None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:37.063020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026b403-eaaa-4b3a-a2ad-09773c7efcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:37.063806 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026b403-eaaa-4b3a-a2ad-09773c7efcfe None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.064007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026b403-eaaa-4b3a-a2ad-09773c7efcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.064283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026b403-eaaa-4b3a-a2ad-09773c7efcfe None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.065011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026b403-eaaa-4b3a-a2ad-09773c7efcfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:37.065321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026b403-eaaa-4b3a-a2ad-09773c7efcfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:37.066729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026b403-eaaa-4b3a-a2ad-09773c7efcfe None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:37.082910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 370/736] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:37.082910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8d64e-969c-44d9-9816-9b39edc447e5 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.082910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8d64e-969c-44d9-9816-9b39edc447e5 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.082910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8d64e-969c-44d9-9816-9b39edc447e5 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.082910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 371/737] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:37.082910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8d64e-969c-44d9-9816-9b39edc447e5 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:37.104259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877e387-2d00-4e23-b58d-2c0f9ac1d9e4 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:37.139403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d45d1-0898-479e-9e52-12226c843e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:37.140226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d45d1-0898-479e-9e52-12226c843e59 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.140508 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d45d1-0898-479e-9e52-12226c843e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.141068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d45d1-0898-479e-9e52-12226c843e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.152140 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d45d1-0898-479e-9e52-12226c843e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:37.154702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d45d1-0898-479e-9e52-12226c843e59 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-2e7d45d1-0898-479e-9e52-12226c843e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:37.174142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:37.175126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:37.176070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:37.176070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:37.176070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:37.176070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:37.176070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 367/738] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:37.176360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d45d1-0898-479e-9e52-12226c843e59 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:37.202888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:37.205872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8d64e-969c-44d9-9816-9b39edc447e5 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:37.241876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd682cf-1d6b-4f10-a0af-c70b45d90322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:37.242562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd682cf-1d6b-4f10-a0af-c70b45d90322 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.242838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd682cf-1d6b-4f10-a0af-c70b45d90322 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.243119 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd682cf-1d6b-4f10-a0af-c70b45d90322 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.243843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd682cf-1d6b-4f10-a0af-c70b45d90322 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:37.245221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd682cf-1d6b-4f10-a0af-c70b45d90322 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:37.259502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:37.270487 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 372/739] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:46:37 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:46:37.272086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14217446-8a61-4426-8aa1-fe1eae555d87 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.272790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14217446-8a61-4426-8aa1-fe1eae555d87 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.273477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14217446-8a61-4426-8aa1-fe1eae555d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.300167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-14217446-8a61-4426-8aa1-fe1eae555d87 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:37.301516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d45d1-0898-479e-9e52-12226c843e59 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:37.301856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-14217446-8a61-4426-8aa1-fe1eae555d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:37.367552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72c778-8fc0-44ae-96a7-a3afa7838b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:37.369412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72c778-8fc0-44ae-96a7-a3afa7838b44 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.369890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72c778-8fc0-44ae-96a7-a3afa7838b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.370358 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72c778-8fc0-44ae-96a7-a3afa7838b44 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.382477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72c778-8fc0-44ae-96a7-a3afa7838b44 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:37.385067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72c778-8fc0-44ae-96a7-a3afa7838b44 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:37.398313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 373/740] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:37.398313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14217446-8a61-4426-8aa1-fe1eae555d87 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-0d72c778-8fc0-44ae-96a7-a3afa7838b44 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:37.412789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:37.413806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:37.414814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:37.414814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:37.414814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:37.414814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:37.414814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:37.414814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:37.414814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:37.414814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:37.414814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:37.414814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:37.419685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 368/741] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:37.428328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72c778-8fc0-44ae-96a7-a3afa7838b44 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:37.537157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14217446-8a61-4426-8aa1-fe1eae555d87 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:37.558112 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72c778-8fc0-44ae-96a7-a3afa7838b44 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:37.567726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:37.569041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:37.576948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1f39e-6db0-4084-895c-5b3fd48ccec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:37.578450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1f39e-6db0-4084-895c-5b3fd48ccec1 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.578807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1f39e-6db0-4084-895c-5b3fd48ccec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.579294 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1f39e-6db0-4084-895c-5b3fd48ccec1 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.580100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1f39e-6db0-4084-895c-5b3fd48ccec1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:37.580480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1f39e-6db0-4084-895c-5b3fd48ccec1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:37.581927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1f39e-6db0-4084-895c-5b3fd48ccec1 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:37.594520 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 374/742] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:37.599033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1f39e-6db0-4084-895c-5b3fd48ccec1 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:37.605052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6593e-9fc7-488d-a71c-19840aee69b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:37.606699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6593e-9fc7-488d-a71c-19840aee69b0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.608163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6593e-9fc7-488d-a71c-19840aee69b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.609200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6593e-9fc7-488d-a71c-19840aee69b0 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.610239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6593e-9fc7-488d-a71c-19840aee69b0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:37.610768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6593e-9fc7-488d-a71c-19840aee69b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:37.612438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6593e-9fc7-488d-a71c-19840aee69b0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:37.640995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 369/743] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 12:46:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:37.641905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6593e-9fc7-488d-a71c-19840aee69b0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:37.713667 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1f39e-6db0-4084-895c-5b3fd48ccec1 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:37.749665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6593e-9fc7-488d-a71c-19840aee69b0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:37.750300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f9d18-2827-4bd4-90ad-e3e6a5d83e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:37.751314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f9d18-2827-4bd4-90ad-e3e6a5d83e64 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.751669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f9d18-2827-4bd4-90ad-e3e6a5d83e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.752090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f9d18-2827-4bd4-90ad-e3e6a5d83e64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.764822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f9d18-2827-4bd4-90ad-e3e6a5d83e64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-273f9d18-2827-4bd4-90ad-e3e6a5d83e64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:37.773343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:37.774402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:37.775267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 375/744] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:37.775967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f9d18-2827-4bd4-90ad-e3e6a5d83e64 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:37.790208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1883ca-be9b-4a64-a0cf-d5034a734662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:37.791237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1883ca-be9b-4a64-a0cf-d5034a734662 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.791516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1883ca-be9b-4a64-a0cf-d5034a734662 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.791888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1883ca-be9b-4a64-a0cf-d5034a734662 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.800000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1883ca-be9b-4a64-a0cf-d5034a734662 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:37.801443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1883ca-be9b-4a64-a0cf-d5034a734662 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:37.803438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-bc1883ca-be9b-4a64-a0cf-d5034a734662 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:37.819168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:37.820280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:37.821264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:37.821264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:37.821264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:37.821264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:37.821264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:37.821264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:37.821264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:37.821264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:37.821264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 370/745] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:37.821264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1883ca-be9b-4a64-a0cf-d5034a734662 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:37.838323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:37.914910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f9d18-2827-4bd4-90ad-e3e6a5d83e64 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:37.949393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104c4c2-3b12-435f-abe6-f80df9f24af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:37.950334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c4c2-3b12-435f-abe6-f80df9f24af5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.950794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c4c2-3b12-435f-abe6-f80df9f24af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.951264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c4c2-3b12-435f-abe6-f80df9f24af5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.951766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1883ca-be9b-4a64-a0cf-d5034a734662 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:37.988629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04310c5f-09bd-44c5-b5dd-6f15558a67bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:37.989681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04310c5f-09bd-44c5-b5dd-6f15558a67bb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:37.989681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04310c5f-09bd-44c5-b5dd-6f15558a67bb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:37.989857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04310c5f-09bd-44c5-b5dd-6f15558a67bb None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:37.990632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04310c5f-09bd-44c5-b5dd-6f15558a67bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:37.990944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04310c5f-09bd-44c5-b5dd-6f15558a67bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:37.992308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04310c5f-09bd-44c5-b5dd-6f15558a67bb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:38.001876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 371/746] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:38.002770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04310c5f-09bd-44c5-b5dd-6f15558a67bb None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:38.028999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104c4c2-3b12-435f-abe6-f80df9f24af5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:38.030520 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104c4c2-3b12-435f-abe6-f80df9f24af5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:38.069890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:38.079737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:38.117532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04310c5f-09bd-44c5-b5dd-6f15558a67bb None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:38.138685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 376/747] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:37 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 12:46:38.138685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6081cd-eb80-4293-8088-10f395748197 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:38.138685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6081cd-eb80-4293-8088-10f395748197 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:38.138685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6081cd-eb80-4293-8088-10f395748197 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:38.138685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 377/748] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:38.139117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6081cd-eb80-4293-8088-10f395748197 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:38.160007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fd78e-bdb4-423f-b633-6e189654e538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:38.160685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fd78e-bdb4-423f-b633-6e189654e538 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:38.160973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fd78e-bdb4-423f-b633-6e189654e538 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:38.161254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fd78e-bdb4-423f-b633-6e189654e538 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:38.162044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fd78e-bdb4-423f-b633-6e189654e538 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:38.162389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fd78e-bdb4-423f-b633-6e189654e538 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:38.163826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fd78e-bdb4-423f-b633-6e189654e538 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:38.177373 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 372/749] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:38.178142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fd78e-bdb4-423f-b633-6e189654e538 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:38.253719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6081cd-eb80-4293-8088-10f395748197 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:38.287172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fd78e-bdb4-423f-b633-6e189654e538 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:38.288620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e68c5-a7ce-4a35-9683-b5152a4b89a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:38.289412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e68c5-a7ce-4a35-9683-b5152a4b89a8 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:38.289695 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e68c5-a7ce-4a35-9683-b5152a4b89a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:38.289969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e68c5-a7ce-4a35-9683-b5152a4b89a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:38.301100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e68c5-a7ce-4a35-9683-b5152a4b89a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:38.302574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e68c5-a7ce-4a35-9683-b5152a4b89a8 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:38.309796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-c67e68c5-a7ce-4a35-9683-b5152a4b89a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:38.320702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:38.321617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:38.322556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:38.322556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:38.322556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:38.322556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:38.322556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 378/750] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:38.323335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96f16-1308-420e-baa9-db0f88499238 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:38.323599 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96f16-1308-420e-baa9-db0f88499238 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:38.323874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96f16-1308-420e-baa9-db0f88499238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:38.329383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:38.330931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa469ea-456f-46b3-9fe1-17ea2c617d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:38.331715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa469ea-456f-46b3-9fe1-17ea2c617d97 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:38.331898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa469ea-456f-46b3-9fe1-17ea2c617d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:38.332155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa469ea-456f-46b3-9fe1-17ea2c617d97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:38.342496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-d6d96f16-1308-420e-baa9-db0f88499238 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:38.342773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa469ea-456f-46b3-9fe1-17ea2c617d97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:38.344253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d6d96f16-1308-420e-baa9-db0f88499238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:38.344797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa469ea-456f-46b3-9fe1-17ea2c617d97 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-eaa469ea-456f-46b3-9fe1-17ea2c617d97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:38.363117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:38.364009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:38.369667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:38.369667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:38.369667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:38.369667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:38.369667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 373/751] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:38.369667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa469ea-456f-46b3-9fe1-17ea2c617d97 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:38.438412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 379/752] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 12:46:38.438412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d96f16-1308-420e-baa9-db0f88499238 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:38.492192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa469ea-456f-46b3-9fe1-17ea2c617d97 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:38.533819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d2beb-c97a-4148-80d8-e4cebcbcabba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:38.534586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d2beb-c97a-4148-80d8-e4cebcbcabba None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:38.534820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d2beb-c97a-4148-80d8-e4cebcbcabba None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:38.535091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d2beb-c97a-4148-80d8-e4cebcbcabba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:38.535936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d2beb-c97a-4148-80d8-e4cebcbcabba None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=122fdcf8f13348cf858555a565e5cf09) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:38.536247 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d2beb-c97a-4148-80d8-e4cebcbcabba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:38.537691 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d2beb-c97a-4148-80d8-e4cebcbcabba None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:38.538748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d96f16-1308-420e-baa9-db0f88499238 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:38.554329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 374/753] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:38 2022] GET /identity/v3/role_assignments?user.id=122fdcf8f13348cf858555a565e5cf09&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:38.554329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d2beb-c97a-4148-80d8-e4cebcbcabba None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:38.584182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:38.592576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf87ba6-d3b8-4412-9611-ad949cd5b192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:38.594524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf87ba6-d3b8-4412-9611-ad949cd5b192 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:38.595117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf87ba6-d3b8-4412-9611-ad949cd5b192 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:38.595398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf87ba6-d3b8-4412-9611-ad949cd5b192 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:38.596200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf87ba6-d3b8-4412-9611-ad949cd5b192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:38.596506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf87ba6-d3b8-4412-9611-ad949cd5b192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:38.599360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:38.600338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf87ba6-d3b8-4412-9611-ad949cd5b192 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:38.618095 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 380/754] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:38.619434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf87ba6-d3b8-4412-9611-ad949cd5b192 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:38.690196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d2beb-c97a-4148-80d8-e4cebcbcabba None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:38.731809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe192a-38a7-47c9-875a-3274023ec505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:38.732777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe192a-38a7-47c9-875a-3274023ec505 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:38.733292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe192a-38a7-47c9-875a-3274023ec505 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:38.733694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe192a-38a7-47c9-875a-3274023ec505 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:38.734782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe192a-38a7-47c9-875a-3274023ec505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:38.735098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe192a-38a7-47c9-875a-3274023ec505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:38.736517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe192a-38a7-47c9-875a-3274023ec505 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:38.756375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf87ba6-d3b8-4412-9611-ad949cd5b192 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:38.757703 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 375/755] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:38.758088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe192a-38a7-47c9-875a-3274023ec505 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:38.800609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d90ac-d50b-4600-9214-38007e674e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:38.801360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d90ac-d50b-4600-9214-38007e674e65 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:38.801641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d90ac-d50b-4600-9214-38007e674e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:38.801915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d90ac-d50b-4600-9214-38007e674e65 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:38.802671 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d90ac-d50b-4600-9214-38007e674e65 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:38.804574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d90ac-d50b-4600-9214-38007e674e65 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:38.823093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 381/756] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:46:38 2022] POST /identity/v3/services => generated 240 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:46:38.824209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d90ac-d50b-4600-9214-38007e674e65 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:38.869356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:38.878354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe192a-38a7-47c9-875a-3274023ec505 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:38.901939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:38.954136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8781bd4-134a-413b-8398-0ddab87bdec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:38.955369 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8781bd4-134a-413b-8398-0ddab87bdec7 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:38.956050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8781bd4-134a-413b-8398-0ddab87bdec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:38.957503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8781bd4-134a-413b-8398-0ddab87bdec7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:38.980593 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d90ac-d50b-4600-9214-38007e674e65 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:38.982790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8781bd4-134a-413b-8398-0ddab87bdec7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:38.984826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8781bd4-134a-413b-8398-0ddab87bdec7 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-e8781bd4-134a-413b-8398-0ddab87bdec7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:39.019478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:39.020599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:39.021547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:39.021547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:39.021547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:39.021547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:39.021547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:39.021547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:39.021547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:39.021547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:39.021547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:39.021547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:39.026784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 376/757] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 12:46:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:39.029365 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8781bd4-134a-413b-8398-0ddab87bdec7 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:39.060331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d9c4a-63e0-4eee-9862-af823daeaadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:39.061159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d9c4a-63e0-4eee-9862-af823daeaadf None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:39.061444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d9c4a-63e0-4eee-9862-af823daeaadf None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:39.061788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d9c4a-63e0-4eee-9862-af823daeaadf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:39.079508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d9c4a-63e0-4eee-9862-af823daeaadf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:39.081006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d9c4a-63e0-4eee-9862-af823daeaadf None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-641d9c4a-63e0-4eee-9862-af823daeaadf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:39.106645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:39.107642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:39.108583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:39.108583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:39.108583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:39.108583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:39.108583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:39.108583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:39.108583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:39.108583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:39.111192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 382/758] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:39.116776 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d9c4a-63e0-4eee-9862-af823daeaadf None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:39.162139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8781bd4-134a-413b-8398-0ddab87bdec7 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:39.204394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0ce68-4931-499d-96b4-cf4382d7e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:39.205164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0ce68-4931-499d-96b4-cf4382d7e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:39.205550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0ce68-4931-499d-96b4-cf4382d7e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:39.205930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0ce68-4931-499d-96b4-cf4382d7e0f2 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:39.206889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0ce68-4931-499d-96b4-cf4382d7e0f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:39.207275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0ce68-4931-499d-96b4-cf4382d7e0f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:39.208874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0ce68-4931-499d-96b4-cf4382d7e0f2 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:39.234568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 377/759] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 12:46:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:39.239904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0ce68-4931-499d-96b4-cf4382d7e0f2 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:39.258007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d9c4a-63e0-4eee-9862-af823daeaadf None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:39.260998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:39.275575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:39.304776 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f36d0-d3d4-40ae-9028-1a1c03ec0ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:39.305831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f36d0-d3d4-40ae-9028-1a1c03ec0ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:39.306120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f36d0-d3d4-40ae-9028-1a1c03ec0ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:39.306433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f36d0-d3d4-40ae-9028-1a1c03ec0ea7 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:39.307239 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f36d0-d3d4-40ae-9028-1a1c03ec0ea7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:39.307554 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f36d0-d3d4-40ae-9028-1a1c03ec0ea7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:39.309590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f36d0-d3d4-40ae-9028-1a1c03ec0ea7 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:39.325974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 383/760] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:39.332676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f36d0-d3d4-40ae-9028-1a1c03ec0ea7 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:39.384067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0ce68-4931-499d-96b4-cf4382d7e0f2 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:39.459825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbfcb9-2267-487f-b656-addc679824d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:39.460569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbfcb9-2267-487f-b656-addc679824d8 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:39.460854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbfcb9-2267-487f-b656-addc679824d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:39.461130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbfcb9-2267-487f-b656-addc679824d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:39.483106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbfcb9-2267-487f-b656-addc679824d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:39.484681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbfcb9-2267-487f-b656-addc679824d8 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-5bdbfcb9-2267-487f-b656-addc679824d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:39.528004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:39.529000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:39.529916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:39.529916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:39.529916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:39.529916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:39.529916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 378/761] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:39.535196 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f36d0-d3d4-40ae-9028-1a1c03ec0ea7 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:39.536086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbfcb9-2267-487f-b656-addc679824d8 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:39.572799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813607b-c2d8-4dc6-bcec-b8fb4bc77cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:39.573855 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813607b-c2d8-4dc6-bcec-b8fb4bc77cbc None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:39.574408 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813607b-c2d8-4dc6-bcec-b8fb4bc77cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:39.574982 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813607b-c2d8-4dc6-bcec-b8fb4bc77cbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:39.576070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813607b-c2d8-4dc6-bcec-b8fb4bc77cbc None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=d00b728319304b1e9ba2f1ee93022187) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:39.576606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813607b-c2d8-4dc6-bcec-b8fb4bc77cbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:39.578489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813607b-c2d8-4dc6-bcec-b8fb4bc77cbc None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:39.584882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 384/762] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:39 2022] GET /identity/v3/role_assignments?user.id=d00b728319304b1e9ba2f1ee93022187&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=0c473678b281495f86f58b077be04e1b => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:39.658799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:39.711867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbfcb9-2267-487f-b656-addc679824d8 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:39.760285 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb624f68-45b3-47bf-8ae8-b79bfca46986 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:39.761290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb624f68-45b3-47bf-8ae8-b79bfca46986 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:39.761617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287946a-8bf8-4519-a8df-b13265ed940e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:39.762026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb624f68-45b3-47bf-8ae8-b79bfca46986 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:39.762323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287946a-8bf8-4519-a8df-b13265ed940e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:39.762619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287946a-8bf8-4519-a8df-b13265ed940e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:39.762902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287946a-8bf8-4519-a8df-b13265ed940e None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:39.763668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287946a-8bf8-4519-a8df-b13265ed940e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:39.763984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287946a-8bf8-4519-a8df-b13265ed940e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:39.765576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 385/763] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:39.770076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287946a-8bf8-4519-a8df-b13265ed940e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:39.771755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b9257-2a2e-41a2-ad88-6433d0b3a75d None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:39.772261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b9257-2a2e-41a2-ad88-6433d0b3a75d None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:39.772758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b9257-2a2e-41a2-ad88-6433d0b3a75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:39.783723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 379/764] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:39.788820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287946a-8bf8-4519-a8df-b13265ed940e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:39.795533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e24b9257-2a2e-41a2-ad88-6433d0b3a75d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:39.797500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e24b9257-2a2e-41a2-ad88-6433d0b3a75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:39.829837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:39.917914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 386/765] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:46:39.933997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287946a-8bf8-4519-a8df-b13265ed940e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:39.951382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b9257-2a2e-41a2-ad88-6433d0b3a75d None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:39.963734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:39.978406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e10c2-14c3-44a7-b9f1-473a5f0eefa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:39.979755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e10c2-14c3-44a7-b9f1-473a5f0eefa1 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:39.980251 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e10c2-14c3-44a7-b9f1-473a5f0eefa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:39.980781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e10c2-14c3-44a7-b9f1-473a5f0eefa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:39.988692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e10c2-14c3-44a7-b9f1-473a5f0eefa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:39.990444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e10c2-14c3-44a7-b9f1-473a5f0eefa1 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-485e10c2-14c3-44a7-b9f1-473a5f0eefa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:40.007708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:40.008733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:40.009645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:40.009645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:40.009645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:40.009645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:40.009645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:40.009645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:40.009645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:40.009645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:40.012621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 380/766] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:40.019209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e10c2-14c3-44a7-b9f1-473a5f0eefa1 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:40.133968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b9257-2a2e-41a2-ad88-6433d0b3a75d None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:40.172761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2962b5-d36b-48bd-bc97-00b20be3fc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:40.173684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2962b5-d36b-48bd-bc97-00b20be3fc6b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:40.173972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2962b5-d36b-48bd-bc97-00b20be3fc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:40.174250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2962b5-d36b-48bd-bc97-00b20be3fc6b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:40.174937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2962b5-d36b-48bd-bc97-00b20be3fc6b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:40.176982 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2962b5-d36b-48bd-bc97-00b20be3fc6b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-ed2962b5-d36b-48bd-bc97-00b20be3fc6b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:40.191572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:40.192526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:40.193367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 387/767] 10.222.0.45 () {56 vars in 1129 bytes} [Tue May 24 12:46:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:40.198308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e10c2-14c3-44a7-b9f1-473a5f0eefa1 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:40.200654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2962b5-d36b-48bd-bc97-00b20be3fc6b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:40.239034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e86e7-4527-4f63-b9bb-ceb234bc3c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:40.239630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e86e7-4527-4f63-b9bb-ceb234bc3c5e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:40.239920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e86e7-4527-4f63-b9bb-ceb234bc3c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:40.240201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e86e7-4527-4f63-b9bb-ceb234bc3c5e None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:40.241000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e86e7-4527-4f63-b9bb-ceb234bc3c5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:40.241313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e86e7-4527-4f63-b9bb-ceb234bc3c5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:40.242812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e86e7-4527-4f63-b9bb-ceb234bc3c5e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:40.271051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 381/768] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:40.271562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:40.272710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e86e7-4527-4f63-b9bb-ceb234bc3c5e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:40.356197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:40.368152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2962b5-d36b-48bd-bc97-00b20be3fc6b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:40.419499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e32601-7874-465e-a748-b9ac49b348e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:40.420191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e32601-7874-465e-a748-b9ac49b348e1 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:40.420481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e32601-7874-465e-a748-b9ac49b348e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:40.420757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e32601-7874-465e-a748-b9ac49b348e1 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:40.422014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e32601-7874-465e-a748-b9ac49b348e1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:40.422350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e32601-7874-465e-a748-b9ac49b348e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:40.423745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e32601-7874-465e-a748-b9ac49b348e1 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:40.429308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 388/769] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 12:46:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:40.435271 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e32601-7874-465e-a748-b9ac49b348e1 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:40.464805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e86e7-4527-4f63-b9bb-ceb234bc3c5e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:40.500753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75536694-5278-484a-a753-27dd375e9f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:40.501580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75536694-5278-484a-a753-27dd375e9f78 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:40.501866 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75536694-5278-484a-a753-27dd375e9f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:40.502165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75536694-5278-484a-a753-27dd375e9f78 None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/05b2edb6fcb7496bb55650080a9da172/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:40.504175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75536694-5278-484a-a753-27dd375e9f78 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=05b2edb6fcb7496bb55650080a9da172, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:40.505628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75536694-5278-484a-a753-27dd375e9f78 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:40.526789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 382/770] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:46:40 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/05b2edb6fcb7496bb55650080a9da172/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 12:46:40.598463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e32601-7874-465e-a748-b9ac49b348e1 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:40.606852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:40.639073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71715c-0823-4b0f-834b-053e1aef01c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:40.639824 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71715c-0823-4b0f-834b-053e1aef01c2 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:40.640115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71715c-0823-4b0f-834b-053e1aef01c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:40.640399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71715c-0823-4b0f-834b-053e1aef01c2 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:40.641140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71715c-0823-4b0f-834b-053e1aef01c2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:40.641454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71715c-0823-4b0f-834b-053e1aef01c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:40.642966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71715c-0823-4b0f-834b-053e1aef01c2 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:40.649303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 389/771] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 12:46:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:40.654855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75536694-5278-484a-a753-27dd375e9f78 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:40.767470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75536694-5278-484a-a753-27dd375e9f78 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:40.768927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:40.803991 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481f927-cc41-44a4-b355-496af3f0e1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:40.804742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481f927-cc41-44a4-b355-496af3f0e1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:40.805033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481f927-cc41-44a4-b355-496af3f0e1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:40.805315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481f927-cc41-44a4-b355-496af3f0e1f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:40.806119 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481f927-cc41-44a4-b355-496af3f0e1f4 None admin] RBAC: Adding query filter params (interface=public, service_id=d13734a33f414d918073032390d33783, region_id=RegionOne) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:40.806649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481f927-cc41-44a4-b355-496af3f0e1f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:40.808029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481f927-cc41-44a4-b355-496af3f0e1f4 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:40.816260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 383/772] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 12:46:40 2022] GET /identity/v3/endpoints?service_id=d13734a33f414d918073032390d33783&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:40.823383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71715c-0823-4b0f-834b-053e1aef01c2 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:40.982945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71715c-0823-4b0f-834b-053e1aef01c2 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:41.033366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83515e43-d2fd-4d4e-bfe6-5d4343ff6b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:41.033973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515e43-d2fd-4d4e-bfe6-5d4343ff6b1c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:41.034265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515e43-d2fd-4d4e-bfe6-5d4343ff6b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:41.034566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515e43-d2fd-4d4e-bfe6-5d4343ff6b1c None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:41.035263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83515e43-d2fd-4d4e-bfe6-5d4343ff6b1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:41.036774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83515e43-d2fd-4d4e-bfe6-5d4343ff6b1c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:41.047838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 390/773] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 12:46:40 2022] GET /identity/v3/services => generated 772 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:41.151829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827def6-c88e-442c-a8ae-2ae95852a59c None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:41.152505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827def6-c88e-442c-a8ae-2ae95852a59c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:41.152994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827def6-c88e-442c-a8ae-2ae95852a59c None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:41.154913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 384/774] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:41.161244 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26b4ef-9693-46d1-8dfe-9735d51ccfb6 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:41.161551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26b4ef-9693-46d1-8dfe-9735d51ccfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:41.161823 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26b4ef-9693-46d1-8dfe-9735d51ccfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:41.187235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6a26b4ef-9693-46d1-8dfe-9735d51ccfb6 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:41.188546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6a26b4ef-9693-46d1-8dfe-9735d51ccfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:41.207757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:41.270404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 391/775] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:41.298068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827def6-c88e-442c-a8ae-2ae95852a59c None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:41.417557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827def6-c88e-442c-a8ae-2ae95852a59c None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:41.462423 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e53834-2a0d-4f7c-87d0-c1821ed39fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:41.463233 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e53834-2a0d-4f7c-87d0-c1821ed39fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:41.463522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e53834-2a0d-4f7c-87d0-c1821ed39fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:41.463801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e53834-2a0d-4f7c-87d0-c1821ed39fb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:41.481979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e53834-2a0d-4f7c-87d0-c1821ed39fb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-c8e53834-2a0d-4f7c-87d0-c1821ed39fb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:41.489961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:41.490915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:41.491801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:41.492940 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 385/776] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:41.503481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26b4ef-9693-46d1-8dfe-9735d51ccfb6 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:41.643293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26b4ef-9693-46d1-8dfe-9735d51ccfb6 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:41.690453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e2d6e-86ab-4029-a055-b7621f83eeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:41.691135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e2d6e-86ab-4029-a055-b7621f83eeca None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:41.691423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e2d6e-86ab-4029-a055-b7621f83eeca None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:41.691701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e2d6e-86ab-4029-a055-b7621f83eeca None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:41.692452 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e2d6e-86ab-4029-a055-b7621f83eeca None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:41.692764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e2d6e-86ab-4029-a055-b7621f83eeca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:41.694134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e2d6e-86ab-4029-a055-b7621f83eeca None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:41.702968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 392/777] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:41.711589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e53834-2a0d-4f7c-87d0-c1821ed39fb5 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:41.748134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:41.772322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa120052-18d2-4786-b768-65cf09dc56ad None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:41.772654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa120052-18d2-4786-b768-65cf09dc56ad None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:41.772889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa120052-18d2-4786-b768-65cf09dc56ad None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:41.774397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 393/778] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:46:41.788468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840773f5-fabf-43c1-ac91-77f917c14c5f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:41.788756 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840773f5-fabf-43c1-ac91-77f917c14c5f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:41.789034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840773f5-fabf-43c1-ac91-77f917c14c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:41.806654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-840773f5-fabf-43c1-ac91-77f917c14c5f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:41.807858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-840773f5-fabf-43c1-ac91-77f917c14c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:41.818102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:41.874802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e53834-2a0d-4f7c-87d0-c1821ed39fb5 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:41.879132 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 394/779] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:41.914265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840773f5-fabf-43c1-ac91-77f917c14c5f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:41.922414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee708f4c-d62f-4623-bb5b-b14bf529245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:41.923244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee708f4c-d62f-4623-bb5b-b14bf529245c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:41.923534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee708f4c-d62f-4623-bb5b-b14bf529245c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:41.923817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee708f4c-d62f-4623-bb5b-b14bf529245c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:41.935712 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee708f4c-d62f-4623-bb5b-b14bf529245c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:41.942307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee708f4c-d62f-4623-bb5b-b14bf529245c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-ee708f4c-d62f-4623-bb5b-b14bf529245c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:41.958390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:41.959329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:41.960224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:41.960224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:41.960224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:41.960224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:41.960224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 386/780] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:41.965227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee708f4c-d62f-4623-bb5b-b14bf529245c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:42.092977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840773f5-fabf-43c1-ac91-77f917c14c5f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:42.105901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:42.131483 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36672d-6a3f-4894-86f5-37c4d8e4a1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:42.132696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36672d-6a3f-4894-86f5-37c4d8e4a1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:42.133188 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36672d-6a3f-4894-86f5-37c4d8e4a1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:42.133666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36672d-6a3f-4894-86f5-37c4d8e4a1a2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:42.134564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36672d-6a3f-4894-86f5-37c4d8e4a1a2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:42.136430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36672d-6a3f-4894-86f5-37c4d8e4a1a2 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-7b36672d-6a3f-4894-86f5-37c4d8e4a1a2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:42.149502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:42.150542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 12:46:42.151398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:42.154380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 395/781] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:41 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:42.159723 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36672d-6a3f-4894-86f5-37c4d8e4a1a2 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:42.169290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee708f4c-d62f-4623-bb5b-b14bf529245c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:42.174756 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:42.273226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36672d-6a3f-4894-86f5-37c4d8e4a1a2 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:42.290983 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf57a66-cd9b-4e3d-ab4c-95a769cea7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:42.291664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf57a66-cd9b-4e3d-ab4c-95a769cea7ad None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:42.291955 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf57a66-cd9b-4e3d-ab4c-95a769cea7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:42.292233 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf57a66-cd9b-4e3d-ab4c-95a769cea7ad None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:42.292966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf57a66-cd9b-4e3d-ab4c-95a769cea7ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:42.293277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf57a66-cd9b-4e3d-ab4c-95a769cea7ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:42.294664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf57a66-cd9b-4e3d-ab4c-95a769cea7ad None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:42.310355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd9260-5959-4a6c-b7ea-f129747de9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:42.312300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9260-5959-4a6c-b7ea-f129747de9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:42.312811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9260-5959-4a6c-b7ea-f129747de9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:42.313307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9260-5959-4a6c-b7ea-f129747de9b4 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:42.314207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd9260-5959-4a6c-b7ea-f129747de9b4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:42.314746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd9260-5959-4a6c-b7ea-f129747de9b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:42.316793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd9260-5959-4a6c-b7ea-f129747de9b4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:42.322446 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 396/782] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:46:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:42.333001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 387/783] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:42.334407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd9260-5959-4a6c-b7ea-f129747de9b4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:42.336116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf57a66-cd9b-4e3d-ab4c-95a769cea7ad None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:42.414536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:42.484083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd9260-5959-4a6c-b7ea-f129747de9b4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:42.535099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf57a66-cd9b-4e3d-ab4c-95a769cea7ad None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:42.570785 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9ed34-c77c-4253-b79f-dec300e3c55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:42.571463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9ed34-c77c-4253-b79f-dec300e3c55b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:42.571746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9ed34-c77c-4253-b79f-dec300e3c55b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:42.572023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9ed34-c77c-4253-b79f-dec300e3c55b None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:42.572745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9ed34-c77c-4253-b79f-dec300e3c55b None admin] RBAC: Adding query filter params (type=network) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:42.573052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9ed34-c77c-4253-b79f-dec300e3c55b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:42.574459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9ed34-c77c-4253-b79f-dec300e3c55b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:42.581167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6981a5-b6af-4e68-a4e1-f38f4b977bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:42.581995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6981a5-b6af-4e68-a4e1-f38f4b977bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:42.582276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6981a5-b6af-4e68-a4e1-f38f4b977bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:42.582577 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6981a5-b6af-4e68-a4e1-f38f4b977bb9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:42.587360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 397/784] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:46:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:42.593224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9ed34-c77c-4253-b79f-dec300e3c55b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:42.597234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6981a5-b6af-4e68-a4e1-f38f4b977bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:42.598747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6981a5-b6af-4e68-a4e1-f38f4b977bb9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:42.606240 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:42.732479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9ed34-c77c-4253-b79f-dec300e3c55b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:42.770479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366073c-03a9-4fa9-9a64-9579c1332bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:42.771071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366073c-03a9-4fa9-9a64-9579c1332bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:42.771359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366073c-03a9-4fa9-9a64-9579c1332bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:42.771636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366073c-03a9-4fa9-9a64-9579c1332bc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:42.772443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366073c-03a9-4fa9-9a64-9579c1332bc5 None admin] RBAC: Adding query filter params (interface=public, service_id=a91a2511ea764e57b3daab60e414d1cc, region_id=RegionOne) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-af6981a5-b6af-4e68-a4e1-f38f4b977bb9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:42.772640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:42.773552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:42.774525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:42.774525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:42.774525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:42.774525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:42.774525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:42.774525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:42.774525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:42.774525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:42.774525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:46:42.774525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:42.775077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366073c-03a9-4fa9-9a64-9579c1332bc5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:42.775155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 388/785] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 12:46:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:42.775694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad3343-e07f-4195-8b1e-c4565b394c9b None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:42.775694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad3343-e07f-4195-8b1e-c4565b394c9b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:42.775975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad3343-e07f-4195-8b1e-c4565b394c9b None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:42.782306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 389/786] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:42.782544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366073c-03a9-4fa9-9a64-9579c1332bc5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:42.782715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad3343-e07f-4195-8b1e-c4565b394c9b None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:42.784934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:42.790835 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 398/787] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 12:46:42 2022] GET /identity/v3/endpoints?service_id=a91a2511ea764e57b3daab60e414d1cc&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:42.792356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5237e8-856a-4903-b9d1-efc6826c13db None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:42.792829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5237e8-856a-4903-b9d1-efc6826c13db None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:42.793313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5237e8-856a-4903-b9d1-efc6826c13db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:42.815295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6b5237e8-856a-4903-b9d1-efc6826c13db None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:42.816785 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6b5237e8-856a-4903-b9d1-efc6826c13db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:42.909744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 399/788] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:42.910909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5237e8-856a-4903-b9d1-efc6826c13db None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:42.931933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad3343-e07f-4195-8b1e-c4565b394c9b None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:42.986437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f682a8d-6783-4eef-a822-d1a1d70eb871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:42.987158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f682a8d-6783-4eef-a822-d1a1d70eb871 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:42.987522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f682a8d-6783-4eef-a822-d1a1d70eb871 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:42.988001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f682a8d-6783-4eef-a822-d1a1d70eb871 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:42.988863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f682a8d-6783-4eef-a822-d1a1d70eb871 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:42.989484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f682a8d-6783-4eef-a822-d1a1d70eb871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:42.990880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f682a8d-6783-4eef-a822-d1a1d70eb871 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:43.011443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 390/789] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 12:46:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:43.016878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f682a8d-6783-4eef-a822-d1a1d70eb871 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:43.044277 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5237e8-856a-4903-b9d1-efc6826c13db None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:43.077863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:43.085282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045d5ba-1830-46fd-b018-61c2b4cbbaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:43.085714 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045d5ba-1830-46fd-b018-61c2b4cbbaff None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.085996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045d5ba-1830-46fd-b018-61c2b4cbbaff None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.086271 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045d5ba-1830-46fd-b018-61c2b4cbbaff None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.086997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045d5ba-1830-46fd-b018-61c2b4cbbaff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:43.088370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045d5ba-1830-46fd-b018-61c2b4cbbaff None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:43.090581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:43.102544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 400/790] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 12:46:42 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:43.103548 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045d5ba-1830-46fd-b018-61c2b4cbbaff None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:43.146209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f682a8d-6783-4eef-a822-d1a1d70eb871 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:43.191313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27683084-48fd-4865-bf3f-05016b9649fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:43.192501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683084-48fd-4865-bf3f-05016b9649fa None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.192973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683084-48fd-4865-bf3f-05016b9649fa None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.193448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683084-48fd-4865-bf3f-05016b9649fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.206225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27683084-48fd-4865-bf3f-05016b9649fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-27683084-48fd-4865-bf3f-05016b9649fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:43.214559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:43.215640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:43.216495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:43.219592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 391/791] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:43.221120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b142529-fc98-41e0-a0cf-8fa39d10b9b4 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.221588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b142529-fc98-41e0-a0cf-8fa39d10b9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.222013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b142529-fc98-41e0-a0cf-8fa39d10b9b4 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.223714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 392/792] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:43.225042 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b142529-fc98-41e0-a0cf-8fa39d10b9b4 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:43.273908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045d5ba-1830-46fd-b018-61c2b4cbbaff None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:43.320767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1751a-c32a-44d3-a3f1-17c79871b207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:43.321862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1751a-c32a-44d3-a3f1-17c79871b207 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.322322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1751a-c32a-44d3-a3f1-17c79871b207 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.322817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1751a-c32a-44d3-a3f1-17c79871b207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.335091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1751a-c32a-44d3-a3f1-17c79871b207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:43.336545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1751a-c32a-44d3-a3f1-17c79871b207 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-2ce1751a-c32a-44d3-a3f1-17c79871b207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:43.355396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:43.356309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:43.357225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:43.357225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:43.357225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:43.357225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:43.359778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 401/793] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:43.361014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4fd37-b59b-479c-a755-738748f35aa0 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.361310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4fd37-b59b-479c-a755-738748f35aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.361588 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4fd37-b59b-479c-a755-738748f35aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.362852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b142529-fc98-41e0-a0cf-8fa39d10b9b4 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:43.380923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:43.394873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-bed4fd37-b59b-479c-a755-738748f35aa0 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:43.396168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bed4fd37-b59b-479c-a755-738748f35aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:43.409370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e84522-b99b-4c5c-b25d-762e59106225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:43.410327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e84522-b99b-4c5c-b25d-762e59106225 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.410864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e84522-b99b-4c5c-b25d-762e59106225 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.411346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e84522-b99b-4c5c-b25d-762e59106225 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.412305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e84522-b99b-4c5c-b25d-762e59106225 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:43.413040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e84522-b99b-4c5c-b25d-762e59106225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:43.414649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e84522-b99b-4c5c-b25d-762e59106225 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:43.424910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 393/794] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:43.426024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e84522-b99b-4c5c-b25d-762e59106225 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:43.444363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:43.473074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 402/795] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:43.474072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4fd37-b59b-479c-a755-738748f35aa0 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:43.545268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e84522-b99b-4c5c-b25d-762e59106225 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:43.582344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65add25c-ad96-44f1-bbff-dd6fd85e95b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:43.582945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65add25c-ad96-44f1-bbff-dd6fd85e95b8 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.583225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65add25c-ad96-44f1-bbff-dd6fd85e95b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.583496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65add25c-ad96-44f1-bbff-dd6fd85e95b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.584213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65add25c-ad96-44f1-bbff-dd6fd85e95b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:43.584514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65add25c-ad96-44f1-bbff-dd6fd85e95b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:43.584754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4fd37-b59b-479c-a755-738748f35aa0 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:43.585874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65add25c-ad96-44f1-bbff-dd6fd85e95b8 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:43.602398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 394/796] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:43.603285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65add25c-ad96-44f1-bbff-dd6fd85e95b8 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:43.621970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff087e4f-bdd6-4a88-aed6-a49353ab127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:43.622944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff087e4f-bdd6-4a88-aed6-a49353ab127e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.623282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff087e4f-bdd6-4a88-aed6-a49353ab127e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.623561 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff087e4f-bdd6-4a88-aed6-a49353ab127e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.635102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff087e4f-bdd6-4a88-aed6-a49353ab127e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:43.636888 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff087e4f-bdd6-4a88-aed6-a49353ab127e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:43.647207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-ff087e4f-bdd6-4a88-aed6-a49353ab127e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:43.658166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:43.659283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:43.660338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:43.660338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:43.660338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:43.660338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:43.661924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 403/797] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:43.662972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff087e4f-bdd6-4a88-aed6-a49353ab127e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:43.695518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:43.710246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65add25c-ad96-44f1-bbff-dd6fd85e95b8 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:43.746194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4bc44a-3fb1-4444-af58-38528cafd18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:43.747018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bc44a-3fb1-4444-af58-38528cafd18c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.747298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bc44a-3fb1-4444-af58-38528cafd18c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.747970 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bc44a-3fb1-4444-af58-38528cafd18c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.761095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4bc44a-3fb1-4444-af58-38528cafd18c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-1a4bc44a-3fb1-4444-af58-38528cafd18c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:43.771121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:43.772041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:43.772877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:43.773521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff087e4f-bdd6-4a88-aed6-a49353ab127e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:43.773596 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 395/798] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:43.774751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212d88c-1a03-4dfd-9c79-29ea6d671bf9 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.775040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212d88c-1a03-4dfd-9c79-29ea6d671bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.775319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212d88c-1a03-4dfd-9c79-29ea6d671bf9 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.776805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 396/799] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:43.778954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212d88c-1a03-4dfd-9c79-29ea6d671bf9 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:43.811882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70b462-2809-4012-94bb-f9c6cbba08cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:43.812678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70b462-2809-4012-94bb-f9c6cbba08cb None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.812962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70b462-2809-4012-94bb-f9c6cbba08cb None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.813247 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70b462-2809-4012-94bb-f9c6cbba08cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.820094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70b462-2809-4012-94bb-f9c6cbba08cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:43.821533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70b462-2809-4012-94bb-f9c6cbba08cb None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-3b70b462-2809-4012-94bb-f9c6cbba08cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:43.835126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:43.836099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:43.836985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:43.836985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:43.836985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:43.836985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:43.836985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:43.836985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:43.836985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:43.836985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:43.840992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 404/800] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:43.841661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70b462-2809-4012-94bb-f9c6cbba08cb None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:43.884312 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:43.890091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212d88c-1a03-4dfd-9c79-29ea6d671bf9 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:43.926057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d311e71-7810-4b4c-9a93-1a5baead2355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:43.926885 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d311e71-7810-4b4c-9a93-1a5baead2355 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.927162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d311e71-7810-4b4c-9a93-1a5baead2355 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.927440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d311e71-7810-4b4c-9a93-1a5baead2355 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.928153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d311e71-7810-4b4c-9a93-1a5baead2355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:43.928455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d311e71-7810-4b4c-9a93-1a5baead2355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:43.930037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d311e71-7810-4b4c-9a93-1a5baead2355 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:43.932573 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:43.947181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 397/801] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:43.947181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cf170-de59-41a4-ac32-5b0873431f0b None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.947181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cf170-de59-41a4-ac32-5b0873431f0b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.947181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cf170-de59-41a4-ac32-5b0873431f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.952108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70b462-2809-4012-94bb-f9c6cbba08cb None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:43.960993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-599cf170-de59-41a4-ac32-5b0873431f0b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:43.962187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-599cf170-de59-41a4-ac32-5b0873431f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:43.988185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b6fa1-3c6d-4dbf-a288-844749d462af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:43.988938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b6fa1-3c6d-4dbf-a288-844749d462af None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:43.989290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b6fa1-3c6d-4dbf-a288-844749d462af None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:43.989575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b6fa1-3c6d-4dbf-a288-844749d462af None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:43.990338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b6fa1-3c6d-4dbf-a288-844749d462af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:43.990648 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b6fa1-3c6d-4dbf-a288-844749d462af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:43.992016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b6fa1-3c6d-4dbf-a288-844749d462af None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:44.006020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 405/802] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:44.006020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b6fa1-3c6d-4dbf-a288-844749d462af None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:44.038323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 398/803] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:44.038323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cf170-de59-41a4-ac32-5b0873431f0b None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:44.112570 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b6fa1-3c6d-4dbf-a288-844749d462af None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:44.138028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:44.148922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6161eee-8bab-4c3b-a55a-ae62d03e987f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:44.149655 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6161eee-8bab-4c3b-a55a-ae62d03e987f None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:44.149941 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6161eee-8bab-4c3b-a55a-ae62d03e987f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:44.150216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6161eee-8bab-4c3b-a55a-ae62d03e987f None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:44.151086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6161eee-8bab-4c3b-a55a-ae62d03e987f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:44.151266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cf170-de59-41a4-ac32-5b0873431f0b None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:44.151440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6161eee-8bab-4c3b-a55a-ae62d03e987f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:44.152831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6161eee-8bab-4c3b-a55a-ae62d03e987f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:44.165268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 406/804] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:44.171240 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6161eee-8bab-4c3b-a55a-ae62d03e987f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:44.183408 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:44.187554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07873dda-d21e-43e4-89fc-37532a82d80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:44.188349 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07873dda-d21e-43e4-89fc-37532a82d80c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:44.188639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07873dda-d21e-43e4-89fc-37532a82d80c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:44.188921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07873dda-d21e-43e4-89fc-37532a82d80c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:44.196073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07873dda-d21e-43e4-89fc-37532a82d80c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:44.197512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07873dda-d21e-43e4-89fc-37532a82d80c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-07873dda-d21e-43e4-89fc-37532a82d80c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:44.212344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:44.213295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 399/805] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:44.215146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07873dda-d21e-43e4-89fc-37532a82d80c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:44.278662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6161eee-8bab-4c3b-a55a-ae62d03e987f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:44.313509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5a93e-23a2-40fb-8635-b48f8fd80f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:44.314312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5a93e-23a2-40fb-8635-b48f8fd80f03 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:44.314610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5a93e-23a2-40fb-8635-b48f8fd80f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:44.314896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5a93e-23a2-40fb-8635-b48f8fd80f03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:44.319870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07873dda-d21e-43e4-89fc-37532a82d80c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:44.325220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5a93e-23a2-40fb-8635-b48f8fd80f03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:44.326705 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5a93e-23a2-40fb-8635-b48f8fd80f03 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-2ab5a93e-23a2-40fb-8635-b48f8fd80f03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:44.345680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:44.346709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:44.347596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:44.347596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:44.347596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:44.347596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:44.347596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 407/806] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:44.351263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5a93e-23a2-40fb-8635-b48f8fd80f03 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:44.360596 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00468f98-f3bb-452c-8650-b32740fa594e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:44.361418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468f98-f3bb-452c-8650-b32740fa594e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:44.361704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468f98-f3bb-452c-8650-b32740fa594e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:44.361982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468f98-f3bb-452c-8650-b32740fa594e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:44.362740 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00468f98-f3bb-452c-8650-b32740fa594e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:44.364160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00468f98-f3bb-452c-8650-b32740fa594e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-00468f98-f3bb-452c-8650-b32740fa594e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:44.378321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:44.379269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:44.385598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 400/807] 10.222.0.45 () {56 vars in 1129 bytes} [Tue May 24 12:46:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:44.386493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00468f98-f3bb-452c-8650-b32740fa594e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:44.408010 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:44.420242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:44.461106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5a93e-23a2-40fb-8635-b48f8fd80f03 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:44.497760 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fad79-00f3-433a-8bc4-cdc5d5610f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:44.498473 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00468f98-f3bb-452c-8650-b32740fa594e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:44.499339 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fad79-00f3-433a-8bc4-cdc5d5610f2f None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:44.499626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fad79-00f3-433a-8bc4-cdc5d5610f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:44.499904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fad79-00f3-433a-8bc4-cdc5d5610f2f None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/122fdcf8f13348cf858555a565e5cf09/roles/1ea1fd1a01f6487aac4236749de7d4e1` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:44.515973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fad79-00f3-433a-8bc4-cdc5d5610f2f None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=122fdcf8f13348cf858555a565e5cf09, role_id=1ea1fd1a01f6487aac4236749de7d4e1)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:44.517551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fad79-00f3-433a-8bc4-cdc5d5610f2f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:44.539872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 408/808] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:46:44 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/122fdcf8f13348cf858555a565e5cf09/roles/1ea1fd1a01f6487aac4236749de7d4e1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:46:44.539872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fad79-00f3-433a-8bc4-cdc5d5610f2f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:44.542463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c486c0-1c19-4768-8362-d52d138a22d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:44.543183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c486c0-1c19-4768-8362-d52d138a22d9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:44.543459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c486c0-1c19-4768-8362-d52d138a22d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:44.543726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c486c0-1c19-4768-8362-d52d138a22d9 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:44.544586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c486c0-1c19-4768-8362-d52d138a22d9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:44.544893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c486c0-1c19-4768-8362-d52d138a22d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:44.546456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c486c0-1c19-4768-8362-d52d138a22d9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:44.569418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 401/809] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 12:46:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:44.571227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c486c0-1c19-4768-8362-d52d138a22d9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:44.659609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fad79-00f3-433a-8bc4-cdc5d5610f2f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:44.673272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:44.679892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:44.704117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3deb69-93e2-4c31-b0cf-a54555d2435a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:44.704831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3deb69-93e2-4c31-b0cf-a54555d2435a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:44.705117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3deb69-93e2-4c31-b0cf-a54555d2435a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:44.705394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3deb69-93e2-4c31-b0cf-a54555d2435a None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:44.706140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3deb69-93e2-4c31-b0cf-a54555d2435a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:44.706552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3deb69-93e2-4c31-b0cf-a54555d2435a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:44.708138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3deb69-93e2-4c31-b0cf-a54555d2435a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:44.714609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c486c0-1c19-4768-8362-d52d138a22d9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:44.725559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 409/810] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:44.727639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3deb69-93e2-4c31-b0cf-a54555d2435a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:44.762629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02aec-e702-4e2d-acc2-608ebcccec82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:44.763544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02aec-e702-4e2d-acc2-608ebcccec82 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:44.764008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02aec-e702-4e2d-acc2-608ebcccec82 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:44.764290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02aec-e702-4e2d-acc2-608ebcccec82 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:44.765062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02aec-e702-4e2d-acc2-608ebcccec82 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:44.765366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02aec-e702-4e2d-acc2-608ebcccec82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:44.766757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02aec-e702-4e2d-acc2-608ebcccec82 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:44.771782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 402/811] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 12:46:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:44.772588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02aec-e702-4e2d-acc2-608ebcccec82 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:44.841608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3deb69-93e2-4c31-b0cf-a54555d2435a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:44.877345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11ef94-3a65-45de-8f01-240f26a0b79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:44.878186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11ef94-3a65-45de-8f01-240f26a0b79d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:44.878508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11ef94-3a65-45de-8f01-240f26a0b79d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:44.878780 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11ef94-3a65-45de-8f01-240f26a0b79d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:44.890519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11ef94-3a65-45de-8f01-240f26a0b79d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:44.891975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11ef94-3a65-45de-8f01-240f26a0b79d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:44.896087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02aec-e702-4e2d-acc2-608ebcccec82 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-ba11ef94-3a65-45de-8f01-240f26a0b79d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:44.909858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:44.910843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:44.911759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:44.911759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:44.911759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:44.911759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:44.911759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 410/812] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:44.912473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11ef94-3a65-45de-8f01-240f26a0b79d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:44.937357 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90345e87-6d13-4b38-9c21-40aa228b0865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:44.938463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90345e87-6d13-4b38-9c21-40aa228b0865 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:44.938924 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90345e87-6d13-4b38-9c21-40aa228b0865 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:44.939451 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90345e87-6d13-4b38-9c21-40aa228b0865 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:44.940170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:44.942802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:44.951355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90345e87-6d13-4b38-9c21-40aa228b0865 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:44.953622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90345e87-6d13-4b38-9c21-40aa228b0865 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-90345e87-6d13-4b38-9c21-40aa228b0865 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:44.969632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:44.970683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:44.971653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:44.971653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:44.971653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:44.971653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:44.971653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:44.971653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:44.971653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:44.971653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:44.971653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 12:46:44.971653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:44.974485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 403/813] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 12:46:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:44.975533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90345e87-6d13-4b38-9c21-40aa228b0865 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:45.031126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11ef94-3a65-45de-8f01-240f26a0b79d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:45.067972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a5e0-5fd0-47fe-9801-871568183fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:45.068653 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a5e0-5fd0-47fe-9801-871568183fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:45.069035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a5e0-5fd0-47fe-9801-871568183fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:45.069318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a5e0-5fd0-47fe-9801-871568183fa8 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:45.070200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a5e0-5fd0-47fe-9801-871568183fa8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:45.070619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a5e0-5fd0-47fe-9801-871568183fa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:45.072310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a5e0-5fd0-47fe-9801-871568183fa8 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:45.083255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 411/814] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 12:46:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:45.084014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a5e0-5fd0-47fe-9801-871568183fa8 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:45.092036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90345e87-6d13-4b38-9c21-40aa228b0865 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:45.131517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3a4c6-1638-4eb7-9300-922cd9999768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:45.132590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3a4c6-1638-4eb7-9300-922cd9999768 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:45.133129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3a4c6-1638-4eb7-9300-922cd9999768 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:45.133726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3a4c6-1638-4eb7-9300-922cd9999768 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:45.134803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3a4c6-1638-4eb7-9300-922cd9999768 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:45.135580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3a4c6-1638-4eb7-9300-922cd9999768 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:45.137243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3a4c6-1638-4eb7-9300-922cd9999768 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:45.150649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 404/815] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:45.151764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3a4c6-1638-4eb7-9300-922cd9999768 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:45.179514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:45.192401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a5e0-5fd0-47fe-9801-871568183fa8 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:45.217236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:45.228146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfc5f9-6760-4408-a80e-9bc060a99b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:45.228827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfc5f9-6760-4408-a80e-9bc060a99b59 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:45.229114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfc5f9-6760-4408-a80e-9bc060a99b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:45.229392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfc5f9-6760-4408-a80e-9bc060a99b59 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:45.230233 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfc5f9-6760-4408-a80e-9bc060a99b59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:45.230594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfc5f9-6760-4408-a80e-9bc060a99b59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:45.232039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfc5f9-6760-4408-a80e-9bc060a99b59 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:45.254721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 412/816] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 12:46:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:45.255733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfc5f9-6760-4408-a80e-9bc060a99b59 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:45.275024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3a4c6-1638-4eb7-9300-922cd9999768 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:45.315993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016247e3-6f40-4fe3-9d04-9976939f1554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:45.317146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016247e3-6f40-4fe3-9d04-9976939f1554 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:45.317654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016247e3-6f40-4fe3-9d04-9976939f1554 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:45.318129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016247e3-6f40-4fe3-9d04-9976939f1554 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:45.319121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016247e3-6f40-4fe3-9d04-9976939f1554 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:45.320697 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016247e3-6f40-4fe3-9d04-9976939f1554 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:45.346114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 405/817] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 12:46:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:46:45.347382 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016247e3-6f40-4fe3-9d04-9976939f1554 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:45.474677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfc5f9-6760-4408-a80e-9bc060a99b59 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:45.491398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016247e3-6f40-4fe3-9d04-9976939f1554 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:45.499867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:45.527783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22bfaa-7ba2-45ff-850e-db5fa640dd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:45.529099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22bfaa-7ba2-45ff-850e-db5fa640dd32 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:45.529532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22bfaa-7ba2-45ff-850e-db5fa640dd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:45.530506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22bfaa-7ba2-45ff-850e-db5fa640dd32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:45.532774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e4ba7-2e01-40fe-ade3-d804752d9ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:45.533527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e4ba7-2e01-40fe-ade3-d804752d9ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:45.533890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e4ba7-2e01-40fe-ade3-d804752d9ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:45.534246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e4ba7-2e01-40fe-ade3-d804752d9ec9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:45.540545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22bfaa-7ba2-45ff-850e-db5fa640dd32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:45.542375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22bfaa-7ba2-45ff-850e-db5fa640dd32 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-fa22bfaa-7ba2-45ff-850e-db5fa640dd32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:45.560805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:45.562634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:45.563636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:45.563636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:45.563636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:45.563636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:45.563636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:45.563636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:45.563636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:45.563636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:45.564534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 413/818] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:45.565491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd593490-4ea0-40fd-9692-1c4e8d60d6b3 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:45.565781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd593490-4ea0-40fd-9692-1c4e8d60d6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:45.566064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd593490-4ea0-40fd-9692-1c4e8d60d6b3 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:45.571826 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 414/819] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:45.572878 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd593490-4ea0-40fd-9692-1c4e8d60d6b3 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:45.584245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:45.627582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e4ba7-2e01-40fe-ade3-d804752d9ec9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:45.629155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e4ba7-2e01-40fe-ade3-d804752d9ec9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:45.725973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd593490-4ea0-40fd-9692-1c4e8d60d6b3 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:45.762463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 406/820] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:46:45.764176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51a4a9-c15b-4039-b3fe-2eafe678b01d None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:45.764737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51a4a9-c15b-4039-b3fe-2eafe678b01d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:45.765214 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51a4a9-c15b-4039-b3fe-2eafe678b01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:45.769568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf37b45-8c33-4f0e-8f47-832a9572bc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:45.770516 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf37b45-8c33-4f0e-8f47-832a9572bc67 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:45.771023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf37b45-8c33-4f0e-8f47-832a9572bc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:45.771498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf37b45-8c33-4f0e-8f47-832a9572bc67 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:45.772506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf37b45-8c33-4f0e-8f47-832a9572bc67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:45.773022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf37b45-8c33-4f0e-8f47-832a9572bc67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:45.774756 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf37b45-8c33-4f0e-8f47-832a9572bc67 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:45.776542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:45.784777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-ff51a4a9-c15b-4039-b3fe-2eafe678b01d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:45.785986 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ff51a4a9-c15b-4039-b3fe-2eafe678b01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:45.790840 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 415/821] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:45.791910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf37b45-8c33-4f0e-8f47-832a9572bc67 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:45.873588 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:45.882143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 407/822] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:45.883378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51a4a9-c15b-4039-b3fe-2eafe678b01d None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:45.913381 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf37b45-8c33-4f0e-8f47-832a9572bc67 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:45.949392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9a93d-cf40-41de-bd71-344d21dd084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:45.950343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a93d-cf40-41de-bd71-344d21dd084a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:45.950481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a93d-cf40-41de-bd71-344d21dd084a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:45.950764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a93d-cf40-41de-bd71-344d21dd084a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:45.961882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9a93d-cf40-41de-bd71-344d21dd084a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:45.963616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9a93d-cf40-41de-bd71-344d21dd084a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-49a9a93d-cf40-41de-bd71-344d21dd084a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:45.981478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:45.982553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:45.983479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:45.983479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:45.983479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:45.983479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:45.983479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 416/823] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:45.983766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9a93d-cf40-41de-bd71-344d21dd084a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:46.009265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51a4a9-c15b-4039-b3fe-2eafe678b01d None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:46.046961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8993f9e-c4eb-4575-ab23-e49820b8045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:46.048084 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8993f9e-c4eb-4575-ab23-e49820b8045e None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:46.048197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8993f9e-c4eb-4575-ab23-e49820b8045e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:46.048447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8993f9e-c4eb-4575-ab23-e49820b8045e None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/d00b728319304b1e9ba2f1ee93022187/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:46.050471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8993f9e-c4eb-4575-ab23-e49820b8045e None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=d00b728319304b1e9ba2f1ee93022187, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:46.051903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8993f9e-c4eb-4575-ab23-e49820b8045e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:46.055075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:46.074549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 408/824] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:46:45 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/d00b728319304b1e9ba2f1ee93022187/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:46:46.074549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8993f9e-c4eb-4575-ab23-e49820b8045e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:46.148010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9a93d-cf40-41de-bd71-344d21dd084a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:46.194128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:46.231224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03b32c-5854-4f2c-945f-adbb25512093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:46.232491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03b32c-5854-4f2c-945f-adbb25512093 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:46.232996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03b32c-5854-4f2c-945f-adbb25512093 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:46.233647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03b32c-5854-4f2c-945f-adbb25512093 None admin] PATH_INFO: `/v3/services/network` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:46.234563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03b32c-5854-4f2c-945f-adbb25512093 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:46.236333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03b32c-5854-4f2c-945f-adbb25512093 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-5d03b32c-5854-4f2c-945f-adbb25512093 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:46.255263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:46.256304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 12:46:46.257128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:46.261041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 417/825] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:45 2022] GET /identity/v3/services/network => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:46.261353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8993f9e-c4eb-4575-ab23-e49820b8045e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:46.266549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03b32c-5854-4f2c-945f-adbb25512093 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:46.297426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ca417-0b4b-40b5-90b1-99094654e3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:46.298110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ca417-0b4b-40b5-90b1-99094654e3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:46.298405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ca417-0b4b-40b5-90b1-99094654e3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:46.298680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ca417-0b4b-40b5-90b1-99094654e3f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:46.299593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ca417-0b4b-40b5-90b1-99094654e3f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:46.299943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ca417-0b4b-40b5-90b1-99094654e3f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:46.301318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ca417-0b4b-40b5-90b1-99094654e3f9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:46.313984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 409/826] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:46.319290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ca417-0b4b-40b5-90b1-99094654e3f9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:46.377104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03b32c-5854-4f2c-945f-adbb25512093 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:46.403889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:46.421543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80335503-444f-42e1-aa6e-e0df4a96a9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:46.422047 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80335503-444f-42e1-aa6e-e0df4a96a9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:46.422434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80335503-444f-42e1-aa6e-e0df4a96a9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:46.422800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80335503-444f-42e1-aa6e-e0df4a96a9a8 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:46.423580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80335503-444f-42e1-aa6e-e0df4a96a9a8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:46.423959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80335503-444f-42e1-aa6e-e0df4a96a9a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:46.425765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80335503-444f-42e1-aa6e-e0df4a96a9a8 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:46.430805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 418/827] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:46:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:46.431646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ca417-0b4b-40b5-90b1-99094654e3f9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:46.435845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80335503-444f-42e1-aa6e-e0df4a96a9a8 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:46.466680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60934e7-dadb-4f8d-9490-3d4862df13dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:46.467947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60934e7-dadb-4f8d-9490-3d4862df13dc None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:46.468448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60934e7-dadb-4f8d-9490-3d4862df13dc None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:46.468920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60934e7-dadb-4f8d-9490-3d4862df13dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:46.476445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60934e7-dadb-4f8d-9490-3d4862df13dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:46.478118 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60934e7-dadb-4f8d-9490-3d4862df13dc None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-f60934e7-dadb-4f8d-9490-3d4862df13dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:46.493218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:46.494104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:46.495043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:46.495043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:46.495043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:46.495043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:46.495043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:46.495043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:46.495043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:46.495043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:46.497459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 410/828] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:46.503658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60934e7-dadb-4f8d-9490-3d4862df13dc None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:46.584166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:46.614381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60934e7-dadb-4f8d-9490-3d4862df13dc None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:46.649730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:46.683481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4aa45-2a2c-4df6-8dea-feda6c721497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:46.684153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4aa45-2a2c-4df6-8dea-feda6c721497 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:46.684435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4aa45-2a2c-4df6-8dea-feda6c721497 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:46.684706 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4aa45-2a2c-4df6-8dea-feda6c721497 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:46.686087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4aa45-2a2c-4df6-8dea-feda6c721497 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:46.686954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4aa45-2a2c-4df6-8dea-feda6c721497 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:46.689191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80335503-444f-42e1-aa6e-e0df4a96a9a8 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:46.692985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4aa45-2a2c-4df6-8dea-feda6c721497 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:46.717622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 411/829] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:46.723021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4aa45-2a2c-4df6-8dea-feda6c721497 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:46.752990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350fb7e-7795-46d9-a3c7-4296c3c16f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:46.755201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350fb7e-7795-46d9-a3c7-4296c3c16f5b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:46.755849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350fb7e-7795-46d9-a3c7-4296c3c16f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:46.757506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350fb7e-7795-46d9-a3c7-4296c3c16f5b None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:46.758808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350fb7e-7795-46d9-a3c7-4296c3c16f5b None admin] RBAC: Adding query filter params (type=network) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:46.760132 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350fb7e-7795-46d9-a3c7-4296c3c16f5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:46.762064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350fb7e-7795-46d9-a3c7-4296c3c16f5b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:46.773700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 419/830] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:46:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:46.780576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350fb7e-7795-46d9-a3c7-4296c3c16f5b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:46.891822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350fb7e-7795-46d9-a3c7-4296c3c16f5b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:46.906634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:46.927960 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe896d-fdc3-4b3f-9fce-4b8fad20c762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:46.929158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe896d-fdc3-4b3f-9fce-4b8fad20c762 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:46.929649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe896d-fdc3-4b3f-9fce-4b8fad20c762 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:46.930189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe896d-fdc3-4b3f-9fce-4b8fad20c762 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:46.931227 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe896d-fdc3-4b3f-9fce-4b8fad20c762 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:46.932811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe896d-fdc3-4b3f-9fce-4b8fad20c762 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:46.966374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 420/831] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 12:46:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:46:46.977959 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4aa45-2a2c-4df6-8dea-feda6c721497 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:47.038786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166f19d-dacb-42f7-bb37-1090f31a9021 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:47.039426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166f19d-dacb-42f7-bb37-1090f31a9021 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:47.039908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166f19d-dacb-42f7-bb37-1090f31a9021 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:47.041646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 421/832] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:46:47.048107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e70697-eea6-4faa-b032-6e2991ea859a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:47.048403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e70697-eea6-4faa-b032-6e2991ea859a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:47.048685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e70697-eea6-4faa-b032-6e2991ea859a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:47.065003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578547a-648f-4b9e-8ae5-59d8e391a5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:47.065709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578547a-648f-4b9e-8ae5-59d8e391a5ef None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:47.065963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578547a-648f-4b9e-8ae5-59d8e391a5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:47.066237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578547a-648f-4b9e-8ae5-59d8e391a5ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:47.067106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578547a-648f-4b9e-8ae5-59d8e391a5ef None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=05b2edb6fcb7496bb55650080a9da172) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:47.067419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578547a-648f-4b9e-8ae5-59d8e391a5ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:47.068876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578547a-648f-4b9e-8ae5-59d8e391a5ef None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:47.078859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-94e70697-eea6-4faa-b032-6e2991ea859a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:47.080076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-94e70697-eea6-4faa-b032-6e2991ea859a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:47.085451 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 412/833] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:46 2022] GET /identity/v3/role_assignments?user.id=05b2edb6fcb7496bb55650080a9da172&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=0c473678b281495f86f58b077be04e1b => generated 608 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:47.210573 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 422/834] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) May 24 12:46:47.244239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578547a-648f-4b9e-8ae5-59d8e391a5ef None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:47.286007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:47.360577 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578547a-648f-4b9e-8ae5-59d8e391a5ef None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:47.397516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50e68-ed6c-4949-86d6-4a8b7ed30889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:47.398823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50e68-ed6c-4949-86d6-4a8b7ed30889 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:47.399315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50e68-ed6c-4949-86d6-4a8b7ed30889 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:47.399796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50e68-ed6c-4949-86d6-4a8b7ed30889 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:47.413823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a50e68-ed6c-4949-86d6-4a8b7ed30889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-69a50e68-ed6c-4949-86d6-4a8b7ed30889 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:47.422186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:47.423168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:47.424034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 413/835] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:46:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:47.431981 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e70697-eea6-4faa-b032-6e2991ea859a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:47.531813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:47.539413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e70697-eea6-4faa-b032-6e2991ea859a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:47.577852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302642d-203b-45a8-b77e-1a1fb89b547a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:47.578940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302642d-203b-45a8-b77e-1a1fb89b547a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:47.579434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302642d-203b-45a8-b77e-1a1fb89b547a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:47.579913 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302642d-203b-45a8-b77e-1a1fb89b547a None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:47.580897 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302642d-203b-45a8-b77e-1a1fb89b547a None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:47.581432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302642d-203b-45a8-b77e-1a1fb89b547a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:47.583045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302642d-203b-45a8-b77e-1a1fb89b547a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:47.592305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 423/836] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:46:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:47.598577 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50e68-ed6c-4949-86d6-4a8b7ed30889 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:47.703989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:47.710715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50e68-ed6c-4949-86d6-4a8b7ed30889 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:47.745632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c4780-80c0-4c06-b731-abca0ae8f417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:47.746836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c4780-80c0-4c06-b731-abca0ae8f417 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:47.747327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c4780-80c0-4c06-b731-abca0ae8f417 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:47.747854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c4780-80c0-4c06-b731-abca0ae8f417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:47.759629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c4780-80c0-4c06-b731-abca0ae8f417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:47.761601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c4780-80c0-4c06-b731-abca0ae8f417 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-786c4780-80c0-4c06-b731-abca0ae8f417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:47.779147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:47.780606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:47.781531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:47.781531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:47.781531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:47.781531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:47.783720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 414/837] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:47.789356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302642d-203b-45a8-b77e-1a1fb89b547a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:47.851971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18344c-9d53-4e28-a910-8626a5f0f47a None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:47.852299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18344c-9d53-4e28-a910-8626a5f0f47a None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:47.852541 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18344c-9d53-4e28-a910-8626a5f0f47a None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:47.854138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 415/838] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:46:47.859864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ceef98-f31f-4532-8b31-d1fae6abc135 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:47.860006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ceef98-f31f-4532-8b31-d1fae6abc135 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:47.860180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ceef98-f31f-4532-8b31-d1fae6abc135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:47.876937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-02ceef98-f31f-4532-8b31-d1fae6abc135 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:47.878120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-02ceef98-f31f-4532-8b31-d1fae6abc135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:47.900427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302642d-203b-45a8-b77e-1a1fb89b547a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:47.943488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c32d03-7390-498b-ba6f-3d6e4d14b795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:47.944783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c32d03-7390-498b-ba6f-3d6e4d14b795 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:47.945513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c32d03-7390-498b-ba6f-3d6e4d14b795 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:47.945974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c32d03-7390-498b-ba6f-3d6e4d14b795 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:47.946911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c32d03-7390-498b-ba6f-3d6e4d14b795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:47.947401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c32d03-7390-498b-ba6f-3d6e4d14b795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:47.948969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c32d03-7390-498b-ba6f-3d6e4d14b795 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:47.954752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 416/839] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:46:47.966485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 424/840] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:47.968164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ceef98-f31f-4532-8b31-d1fae6abc135 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:47.976250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c32d03-7390-498b-ba6f-3d6e4d14b795 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:48.006718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:48.055528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:48.083885 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ceef98-f31f-4532-8b31-d1fae6abc135 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:48.100852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c32d03-7390-498b-ba6f-3d6e4d14b795 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:48.131691 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3ab97-6646-4dcf-bb15-a56dfe05dfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:48.132512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3ab97-6646-4dcf-bb15-a56dfe05dfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:48.132795 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3ab97-6646-4dcf-bb15-a56dfe05dfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:48.133070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3ab97-6646-4dcf-bb15-a56dfe05dfa7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:48.144027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3ab97-6646-4dcf-bb15-a56dfe05dfa7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:48.145468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3ab97-6646-4dcf-bb15-a56dfe05dfa7 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:48.145969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda6c1c-cf0d-4cb0-96ec-923c4b157a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:48.147111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda6c1c-cf0d-4cb0-96ec-923c4b157a44 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:48.147757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda6c1c-cf0d-4cb0-96ec-923c4b157a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:48.148652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda6c1c-cf0d-4cb0-96ec-923c4b157a44 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:48.149514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda6c1c-cf0d-4cb0-96ec-923c4b157a44 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:48.151101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda6c1c-cf0d-4cb0-96ec-923c4b157a44 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-bbb3ab97-6646-4dcf-bb15-a56dfe05dfa7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:48.161129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:48.162100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:48.168110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:48.168110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:48.168110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:48.168110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:48.168110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:48.168110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:48.168110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:48.168110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:48.168110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:46:48.168110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:48.168110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 417/841] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 12:46:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-9cda6c1c-cf0d-4cb0-96ec-923c4b157a44 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:48.169096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:48.172637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 12:46:48.175127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:48.177991 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3ab97-6646-4dcf-bb15-a56dfe05dfa7 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:48.178141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 425/842] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:48.182899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda6c1c-cf0d-4cb0-96ec-923c4b157a44 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:48.278534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3ab97-6646-4dcf-bb15-a56dfe05dfa7 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:48.293761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:48.315733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82112429-865a-4a79-bd8b-f6c68f6b1cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:48.316270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82112429-865a-4a79-bd8b-f6c68f6b1cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:48.316561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82112429-865a-4a79-bd8b-f6c68f6b1cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:48.316842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82112429-865a-4a79-bd8b-f6c68f6b1cb0 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:48.317690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82112429-865a-4a79-bd8b-f6c68f6b1cb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:48.318004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82112429-865a-4a79-bd8b-f6c68f6b1cb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:48.319556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82112429-865a-4a79-bd8b-f6c68f6b1cb0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:48.320075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda6c1c-cf0d-4cb0-96ec-923c4b157a44 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:48.328196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:48.346139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 418/843] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 12:46:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:48.347098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82112429-865a-4a79-bd8b-f6c68f6b1cb0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:48.364895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7d9fe-87d5-4645-b20b-bc119fe97c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:48.366017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7d9fe-87d5-4645-b20b-bc119fe97c1c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:48.366857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7d9fe-87d5-4645-b20b-bc119fe97c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:48.367385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7d9fe-87d5-4645-b20b-bc119fe97c1c None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:48.368320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7d9fe-87d5-4645-b20b-bc119fe97c1c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:48.368830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7d9fe-87d5-4645-b20b-bc119fe97c1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:48.371249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7d9fe-87d5-4645-b20b-bc119fe97c1c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:48.383671 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 426/844] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:48.385219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7d9fe-87d5-4645-b20b-bc119fe97c1c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:48.475448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82112429-865a-4a79-bd8b-f6c68f6b1cb0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:48.510108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7d9fe-87d5-4645-b20b-bc119fe97c1c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:48.517295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24822639-1c42-4c7b-8409-2b988999a12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:48.518342 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24822639-1c42-4c7b-8409-2b988999a12b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:48.518830 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24822639-1c42-4c7b-8409-2b988999a12b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:48.519288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24822639-1c42-4c7b-8409-2b988999a12b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:48.556036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c1e52-0f6a-4b33-868c-e13908838d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:48.556751 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c1e52-0f6a-4b33-868c-e13908838d65 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:48.557108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c1e52-0f6a-4b33-868c-e13908838d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:48.557462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c1e52-0f6a-4b33-868c-e13908838d65 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:48.558627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c1e52-0f6a-4b33-868c-e13908838d65 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:48.559865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c1e52-0f6a-4b33-868c-e13908838d65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:48.561857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c1e52-0f6a-4b33-868c-e13908838d65 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:48.570326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 427/845] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:48 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:48.598797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:48.607606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24822639-1c42-4c7b-8409-2b988999a12b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:48.611024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24822639-1c42-4c7b-8409-2b988999a12b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:48.685927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee60a2-a912-4e08-be64-7834c813d2ef None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:48.686659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee60a2-a912-4e08-be64-7834c813d2ef None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:48.687181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee60a2-a912-4e08-be64-7834c813d2ef None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:48.690066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 428/846] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:46:48.697650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dc382-e653-487e-a406-9341898e7661 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:48.697963 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dc382-e653-487e-a406-9341898e7661 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:48.698204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dc382-e653-487e-a406-9341898e7661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:48.725455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-781dc382-e653-487e-a406-9341898e7661 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:48.726726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-781dc382-e653-487e-a406-9341898e7661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:48.730635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:48.742634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 419/847] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:46:48.748385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24822639-1c42-4c7b-8409-2b988999a12b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:48.808575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 429/848] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:46:48.831965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781dc382-e653-487e-a406-9341898e7661 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:48.856951 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:48.864848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24822639-1c42-4c7b-8409-2b988999a12b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:48.906369 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457af15-e8b5-4b62-8942-991dd79d13bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:48.907606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457af15-e8b5-4b62-8942-991dd79d13bf None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:48.907894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457af15-e8b5-4b62-8942-991dd79d13bf None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:48.908167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457af15-e8b5-4b62-8942-991dd79d13bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:48.922683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457af15-e8b5-4b62-8942-991dd79d13bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:48.924150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457af15-e8b5-4b62-8942-991dd79d13bf None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-8457af15-e8b5-4b62-8942-991dd79d13bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:48.948057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:48.949037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:48.949939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:48.949939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:48.949939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:48.949939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:48.949939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 420/849] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:48.954668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457af15-e8b5-4b62-8942-991dd79d13bf None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:48.958736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781dc382-e653-487e-a406-9341898e7661 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:48.998435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e950d8-1fcf-40f5-a03d-0797c7b7df1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:49.000242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e950d8-1fcf-40f5-a03d-0797c7b7df1a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.000733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e950d8-1fcf-40f5-a03d-0797c7b7df1a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.001209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e950d8-1fcf-40f5-a03d-0797c7b7df1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.010198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:49.014041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e950d8-1fcf-40f5-a03d-0797c7b7df1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-72e950d8-1fcf-40f5-a03d-0797c7b7df1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:49.024153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:49.025096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:46:49.025938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:49.028800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 430/850] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:49.036886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e950d8-1fcf-40f5-a03d-0797c7b7df1a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:49.060976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457af15-e8b5-4b62-8942-991dd79d13bf None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:49.098700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c462fa-2f0b-4cc8-b7e6-3725259e7633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:49.099353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c462fa-2f0b-4cc8-b7e6-3725259e7633 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.099705 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c462fa-2f0b-4cc8-b7e6-3725259e7633 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.099987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c462fa-2f0b-4cc8-b7e6-3725259e7633 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.100720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c462fa-2f0b-4cc8-b7e6-3725259e7633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:49.101032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c462fa-2f0b-4cc8-b7e6-3725259e7633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:49.102456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c462fa-2f0b-4cc8-b7e6-3725259e7633 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:49.118682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 421/851] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:49.119910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c462fa-2f0b-4cc8-b7e6-3725259e7633 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:49.132124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:49.165500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e950d8-1fcf-40f5-a03d-0797c7b7df1a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:49.216111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f33fe-6718-4162-8560-03702c9627c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:49.217868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f33fe-6718-4162-8560-03702c9627c8 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.218331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f33fe-6718-4162-8560-03702c9627c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.218752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f33fe-6718-4162-8560-03702c9627c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.220257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f33fe-6718-4162-8560-03702c9627c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:49.220799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f33fe-6718-4162-8560-03702c9627c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:49.222338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f33fe-6718-4162-8560-03702c9627c8 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:49.239056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c462fa-2f0b-4cc8-b7e6-3725259e7633 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:49.239242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 431/852] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:46:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:49.241643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f33fe-6718-4162-8560-03702c9627c8 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:49.280810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1d1d3-0a19-45cb-98a9-bbdbc69fa212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:49.282555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1d1d3-0a19-45cb-98a9-bbdbc69fa212 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.283013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1d1d3-0a19-45cb-98a9-bbdbc69fa212 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.283439 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1d1d3-0a19-45cb-98a9-bbdbc69fa212 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.297397 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1d1d3-0a19-45cb-98a9-bbdbc69fa212 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:49.299839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1d1d3-0a19-45cb-98a9-bbdbc69fa212 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:49.314935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-18b1d1d3-0a19-45cb-98a9-bbdbc69fa212 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:49.320160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:49.321056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:49.321957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:49.321957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:49.321957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:49.321957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:49.321957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:49.321957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:49.321957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:49.321957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:49.326629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 422/853] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:49.333674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1d1d3-0a19-45cb-98a9-bbdbc69fa212 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:49.371230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f33fe-6718-4162-8560-03702c9627c8 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:49.413133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb57f90-c447-456a-ab5d-0c5d3f01d8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:49.414163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb57f90-c447-456a-ab5d-0c5d3f01d8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.414651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb57f90-c447-456a-ab5d-0c5d3f01d8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.415204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb57f90-c447-456a-ab5d-0c5d3f01d8a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.427384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:49.430196 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb57f90-c447-456a-ab5d-0c5d3f01d8a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:49.431717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb57f90-c447-456a-ab5d-0c5d3f01d8a1 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-cfb57f90-c447-456a-ab5d-0c5d3f01d8a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:49.451026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:49.452611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:49.453501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:49.453501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:49.453501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:49.453501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:49.457441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 432/854] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:49.458947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb57f90-c447-456a-ab5d-0c5d3f01d8a1 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:49.468033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1d1d3-0a19-45cb-98a9-bbdbc69fa212 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:49.510170 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3410fb-1fa4-454d-8187-a5a276062c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:49.511797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3410fb-1fa4-454d-8187-a5a276062c36 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.512866 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3410fb-1fa4-454d-8187-a5a276062c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.513360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3410fb-1fa4-454d-8187-a5a276062c36 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.514992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3410fb-1fa4-454d-8187-a5a276062c36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:49.516787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3410fb-1fa4-454d-8187-a5a276062c36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:49.519017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3410fb-1fa4-454d-8187-a5a276062c36 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:49.538340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 423/855] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:49.538340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe861d-0f46-451f-9b64-a0b8c1b77e14 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.538340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe861d-0f46-451f-9b64-a0b8c1b77e14 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.538648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe861d-0f46-451f-9b64-a0b8c1b77e14 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.540956 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 424/856] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:46:49.541637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe861d-0f46-451f-9b64-a0b8c1b77e14 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:49.569636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb57f90-c447-456a-ab5d-0c5d3f01d8a1 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:49.583405 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:49.607462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a75f1-97ea-4dd3-9717-b8b6ccec3ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:49.608162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a75f1-97ea-4dd3-9717-b8b6ccec3ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.608445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a75f1-97ea-4dd3-9717-b8b6ccec3ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.608719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a75f1-97ea-4dd3-9717-b8b6ccec3ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.609451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a75f1-97ea-4dd3-9717-b8b6ccec3ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:49.609744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a75f1-97ea-4dd3-9717-b8b6ccec3ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:49.611167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a75f1-97ea-4dd3-9717-b8b6ccec3ed5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:49.627823 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 433/857] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:49.627823 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65beb3b-36b3-410b-bcc4-d9505777e5ec None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.627823 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65beb3b-36b3-410b-bcc4-d9505777e5ec None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.627823 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65beb3b-36b3-410b-bcc4-d9505777e5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.641476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f65beb3b-36b3-410b-bcc4-d9505777e5ec None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:49.642699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f65beb3b-36b3-410b-bcc4-d9505777e5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:49.669205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe861d-0f46-451f-9b64-a0b8c1b77e14 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:49.696833 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:49.706089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c2613-20c1-4499-b80c-2a36e44d0905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:49.706779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c2613-20c1-4499-b80c-2a36e44d0905 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.707058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c2613-20c1-4499-b80c-2a36e44d0905 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.707339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c2613-20c1-4499-b80c-2a36e44d0905 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.708171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c2613-20c1-4499-b80c-2a36e44d0905 None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=122fdcf8f13348cf858555a565e5cf09) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:49.708484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c2613-20c1-4499-b80c-2a36e44d0905 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:49.709922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c2613-20c1-4499-b80c-2a36e44d0905 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:49.712454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 434/858] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:49.713485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65beb3b-36b3-410b-bcc4-d9505777e5ec None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:49.721593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 425/859] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:49 2022] GET /identity/v3/role_assignments?user.id=122fdcf8f13348cf858555a565e5cf09&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:49.736483 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c2613-20c1-4499-b80c-2a36e44d0905 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:49.845613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65beb3b-36b3-410b-bcc4-d9505777e5ec None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:49.849884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:49.873050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c2613-20c1-4499-b80c-2a36e44d0905 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:49.888983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcbcc1-afdf-4038-bb75-5753b25dcfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:49.890768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcbcc1-afdf-4038-bb75-5753b25dcfe9 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.891264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcbcc1-afdf-4038-bb75-5753b25dcfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.891739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcbcc1-afdf-4038-bb75-5753b25dcfe9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.900121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcbcc1-afdf-4038-bb75-5753b25dcfe9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:49.901947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcbcc1-afdf-4038-bb75-5753b25dcfe9 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-c8fcbcc1-afdf-4038-bb75-5753b25dcfe9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:49.918164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:49.919222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:49.920163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:49.920163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:49.920163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:49.920163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:49.920163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:49.920163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:49.920163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:49.920163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:49.920163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 12:46:49.920163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:49.920760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f7eee-8206-48a8-a881-90447d31dec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:49.920760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f7eee-8206-48a8-a881-90447d31dec0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:49.920760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f7eee-8206-48a8-a881-90447d31dec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:49.920760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f7eee-8206-48a8-a881-90447d31dec0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:49.921041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f7eee-8206-48a8-a881-90447d31dec0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:49.921792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f7eee-8206-48a8-a881-90447d31dec0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:49.923915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 435/860] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:46:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:49.930218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcbcc1-afdf-4038-bb75-5753b25dcfe9 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-b06f7eee-8206-48a8-a881-90447d31dec0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:49.941292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:49.942250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 12:46:49.943354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:49.945997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 426/861] 10.222.0.45 () {56 vars in 1132 bytes} [Tue May 24 12:46:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:49.952707 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f7eee-8206-48a8-a881-90447d31dec0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:50.019342 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:50.056842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcbcc1-afdf-4038-bb75-5753b25dcfe9 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:50.084387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f7eee-8206-48a8-a881-90447d31dec0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:50.104372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33678306-cb23-4be0-b238-0add407b4059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:50.105039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678306-cb23-4be0-b238-0add407b4059 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:50.105332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678306-cb23-4be0-b238-0add407b4059 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:50.105609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678306-cb23-4be0-b238-0add407b4059 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:50.106489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33678306-cb23-4be0-b238-0add407b4059 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:50.107338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33678306-cb23-4be0-b238-0add407b4059 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:50.109136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33678306-cb23-4be0-b238-0add407b4059 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:50.122092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f1ddc-6071-435e-892c-037605e1c4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:50.122902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f1ddc-6071-435e-892c-037605e1c4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:50.123196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f1ddc-6071-435e-892c-037605e1c4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:50.123472 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f1ddc-6071-435e-892c-037605e1c4a9 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:50.124220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f1ddc-6071-435e-892c-037605e1c4a9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:50.124530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f1ddc-6071-435e-892c-037605e1c4a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:50.125895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f1ddc-6071-435e-892c-037605e1c4a9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:50.131116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 427/862] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 12:46:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:50.132764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 436/863] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 12:46:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:50.136855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f1ddc-6071-435e-892c-037605e1c4a9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:50.140847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33678306-cb23-4be0-b238-0add407b4059 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:50.174525 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:50.264013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33678306-cb23-4be0-b238-0add407b4059 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:50.268351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f1ddc-6071-435e-892c-037605e1c4a9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:50.294827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:50.309594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5553402-aee3-4740-9c1f-0bce9abdc92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:50.310793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5553402-aee3-4740-9c1f-0bce9abdc92a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:50.311322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5553402-aee3-4740-9c1f-0bce9abdc92a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:50.311585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6adab-1639-4b25-8006-20f80ed0fdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:50.312054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5553402-aee3-4740-9c1f-0bce9abdc92a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:50.312259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6adab-1639-4b25-8006-20f80ed0fdfb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:50.312552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6adab-1639-4b25-8006-20f80ed0fdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:50.312995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6adab-1639-4b25-8006-20f80ed0fdfb None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:50.313852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6adab-1639-4b25-8006-20f80ed0fdfb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:50.314127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6adab-1639-4b25-8006-20f80ed0fdfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:50.315535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6adab-1639-4b25-8006-20f80ed0fdfb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:50.326320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 428/864] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 12:46:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:50.411111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5553402-aee3-4740-9c1f-0bce9abdc92a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:50.412633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5553402-aee3-4740-9c1f-0bce9abdc92a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:50.440197 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:50.525909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 437/865] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:50 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 12:46:50.532288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6adab-1639-4b25-8006-20f80ed0fdfb None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:50.645757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6adab-1639-4b25-8006-20f80ed0fdfb None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:50.681913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d64fd-b028-4d06-af0f-6f0e23bed958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:50.683033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d64fd-b028-4d06-af0f-6f0e23bed958 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:50.683506 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d64fd-b028-4d06-af0f-6f0e23bed958 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:50.683968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d64fd-b028-4d06-af0f-6f0e23bed958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:50.696195 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d64fd-b028-4d06-af0f-6f0e23bed958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:50.697931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d64fd-b028-4d06-af0f-6f0e23bed958 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-9e7d64fd-b028-4d06-af0f-6f0e23bed958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:50.716748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:50.718303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:50.719287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:50.719287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:50.719287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:50.719287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:50.721495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 429/866] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:50.727232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5553402-aee3-4740-9c1f-0bce9abdc92a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:50.842616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5553402-aee3-4740-9c1f-0bce9abdc92a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:50.868597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:50.881044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc7f61-7bc1-486c-9cf3-fc69fea8218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:50.881768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc7f61-7bc1-486c-9cf3-fc69fea8218e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:50.882059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc7f61-7bc1-486c-9cf3-fc69fea8218e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:50.882402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc7f61-7bc1-486c-9cf3-fc69fea8218e None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:50.883131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc7f61-7bc1-486c-9cf3-fc69fea8218e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:50.883440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc7f61-7bc1-486c-9cf3-fc69fea8218e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:50.884814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc7f61-7bc1-486c-9cf3-fc69fea8218e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:50.897131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 438/867] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:50.902650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d64fd-b028-4d06-af0f-6f0e23bed958 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:50.915400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75107d8-02ce-412c-8d72-8e3e3381da61 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:50.915692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75107d8-02ce-412c-8d72-8e3e3381da61 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:50.915978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75107d8-02ce-412c-8d72-8e3e3381da61 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:50.917602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 439/868] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:50.922893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c145b7-6859-43a4-aac3-052a2d9ee272 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:50.923194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c145b7-6859-43a4-aac3-052a2d9ee272 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:50.923479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c145b7-6859-43a4-aac3-052a2d9ee272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:50.943463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-46c145b7-6859-43a4-aac3-052a2d9ee272 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:50.944594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-46c145b7-6859-43a4-aac3-052a2d9ee272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:50.962860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:51.034664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d64fd-b028-4d06-af0f-6f0e23bed958 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:51.038171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 440/869] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:46:51.058972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c145b7-6859-43a4-aac3-052a2d9ee272 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:51.084229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5f854-830c-4a77-9de4-f1f8ba5430c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:51.085472 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5f854-830c-4a77-9de4-f1f8ba5430c3 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:51.086036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5f854-830c-4a77-9de4-f1f8ba5430c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:51.086416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5f854-830c-4a77-9de4-f1f8ba5430c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:51.096274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5f854-830c-4a77-9de4-f1f8ba5430c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:51.097732 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5f854-830c-4a77-9de4-f1f8ba5430c3 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-e5f5f854-830c-4a77-9de4-f1f8ba5430c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:51.117484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:51.118484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:51.119412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:51.119412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:51.119412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:51.119412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:51.119412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:51.119412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:51.119412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:51.119412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:51.119412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 430/870] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:51.126320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5f854-830c-4a77-9de4-f1f8ba5430c3 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:51.176732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c145b7-6859-43a4-aac3-052a2d9ee272 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:51.191345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:51.216589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180f9f0-5318-421e-82d7-6519fcd0e3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:51.218011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180f9f0-5318-421e-82d7-6519fcd0e3e4 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:51.218444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180f9f0-5318-421e-82d7-6519fcd0e3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:51.219418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180f9f0-5318-421e-82d7-6519fcd0e3e4 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:51.219825 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180f9f0-5318-421e-82d7-6519fcd0e3e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:51.221305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180f9f0-5318-421e-82d7-6519fcd0e3e4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:51.242325 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 441/871] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:46:51 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:46:51.253310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:51.261123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5f854-830c-4a77-9de4-f1f8ba5430c3 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:51.308300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd76cc-e2da-4f7f-be72-3d5dd20d5a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:51.309151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd76cc-e2da-4f7f-be72-3d5dd20d5a76 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:51.309441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd76cc-e2da-4f7f-be72-3d5dd20d5a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:51.309721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd76cc-e2da-4f7f-be72-3d5dd20d5a76 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:51.310610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd76cc-e2da-4f7f-be72-3d5dd20d5a76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:51.310925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd76cc-e2da-4f7f-be72-3d5dd20d5a76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:51.312440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd76cc-e2da-4f7f-be72-3d5dd20d5a76 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:51.326702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 431/872] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:51.332903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180f9f0-5318-421e-82d7-6519fcd0e3e4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:51.457444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180f9f0-5318-421e-82d7-6519fcd0e3e4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:51.495158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76498554-a8a6-4ace-bd3b-f4cc88428649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:51.495772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76498554-a8a6-4ace-bd3b-f4cc88428649 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:51.496046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76498554-a8a6-4ace-bd3b-f4cc88428649 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:51.496323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76498554-a8a6-4ace-bd3b-f4cc88428649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:51.497162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76498554-a8a6-4ace-bd3b-f4cc88428649 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=d00b728319304b1e9ba2f1ee93022187) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:51.497475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76498554-a8a6-4ace-bd3b-f4cc88428649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:51.499006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76498554-a8a6-4ace-bd3b-f4cc88428649 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:51.505887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 442/873] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:51 2022] GET /identity/v3/role_assignments?user.id=d00b728319304b1e9ba2f1ee93022187&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=0c473678b281495f86f58b077be04e1b => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:52.123261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8faf89-b460-4c6f-8303-92a6c9fd0ad0 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:52.124169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8faf89-b460-4c6f-8303-92a6c9fd0ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:52.124369 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8faf89-b460-4c6f-8303-92a6c9fd0ad0 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:52.125861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 432/874] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:52.131239 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada90289-c3b0-44fc-9aff-15ff844914ee None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:52.131705 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada90289-c3b0-44fc-9aff-15ff844914ee None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:52.131786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada90289-c3b0-44fc-9aff-15ff844914ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:52.149484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ada90289-c3b0-44fc-9aff-15ff844914ee None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:52.150736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ada90289-c3b0-44fc-9aff-15ff844914ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:52.182112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:52.224052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 443/875] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:52.246729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8faf89-b460-4c6f-8303-92a6c9fd0ad0 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:52.354318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8faf89-b460-4c6f-8303-92a6c9fd0ad0 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:52.398920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4588c84-cf1a-4076-8637-7ff23df7afba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:52.402572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4588c84-cf1a-4076-8637-7ff23df7afba None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:52.402572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4588c84-cf1a-4076-8637-7ff23df7afba None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:52.402572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4588c84-cf1a-4076-8637-7ff23df7afba None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:52.415471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4588c84-cf1a-4076-8637-7ff23df7afba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-f4588c84-cf1a-4076-8637-7ff23df7afba None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:52.424230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:52.425285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:52.426212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 433/876] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 12:46:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:52.433709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada90289-c3b0-44fc-9aff-15ff844914ee None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:52.541956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada90289-c3b0-44fc-9aff-15ff844914ee None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:52.578129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88356d7-1fe3-4428-88c3-0262ec4c086b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:52.578847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88356d7-1fe3-4428-88c3-0262ec4c086b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:52.579134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88356d7-1fe3-4428-88c3-0262ec4c086b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:52.579412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88356d7-1fe3-4428-88c3-0262ec4c086b None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:52.580172 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88356d7-1fe3-4428-88c3-0262ec4c086b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:52.580473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88356d7-1fe3-4428-88c3-0262ec4c086b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:52.581869 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88356d7-1fe3-4428-88c3-0262ec4c086b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:52.590717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 444/877] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 12:46:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:52.596731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4588c84-cf1a-4076-8637-7ff23df7afba None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:52.616695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:52.629107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958004f-1151-482a-bc86-150d7427777e None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:52.629722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958004f-1151-482a-bc86-150d7427777e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:52.630208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958004f-1151-482a-bc86-150d7427777e None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:52.632230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 445/878] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:46:52.637457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360aa0a-f474-41a2-92ea-084dc6c7f313 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:52.637972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360aa0a-f474-41a2-92ea-084dc6c7f313 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:52.638489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360aa0a-f474-41a2-92ea-084dc6c7f313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:52.659733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f360aa0a-f474-41a2-92ea-084dc6c7f313 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:52.661115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f360aa0a-f474-41a2-92ea-084dc6c7f313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:52.691749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:52.718868 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4588c84-cf1a-4076-8637-7ff23df7afba None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:52.741866 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 446/879] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:46:52.760058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c08535-a97f-4306-ac25-d347852c42d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:52.761127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c08535-a97f-4306-ac25-d347852c42d4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:52.761417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c08535-a97f-4306-ac25-d347852c42d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:52.761701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c08535-a97f-4306-ac25-d347852c42d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:52.765676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360aa0a-f474-41a2-92ea-084dc6c7f313 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:52.774478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c08535-a97f-4306-ac25-d347852c42d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:52.776099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c08535-a97f-4306-ac25-d347852c42d4 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-77c08535-a97f-4306-ac25-d347852c42d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:52.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:52.796734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:52.797653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:52.797653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:52.797653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:52.797653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:52.797653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 434/880] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:52.803830 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c08535-a97f-4306-ac25-d347852c42d4 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:52.870788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:52.884797 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360aa0a-f474-41a2-92ea-084dc6c7f313 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:52.910361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c08535-a97f-4306-ac25-d347852c42d4 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:52.928186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c742d1-4c7f-4e9c-8b8f-b65927f883a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:52.929019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c742d1-4c7f-4e9c-8b8f-b65927f883a8 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:52.929315 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c742d1-4c7f-4e9c-8b8f-b65927f883a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:52.929599 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c742d1-4c7f-4e9c-8b8f-b65927f883a8 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:52.934048 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c742d1-4c7f-4e9c-8b8f-b65927f883a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:52.934048 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c742d1-4c7f-4e9c-8b8f-b65927f883a8 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:52.955954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 447/881] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:46:52 2022] POST /identity/v3/services => generated 269 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:46:52.969353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc86958-0c17-4485-956c-85f5f6e5712f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:52.970091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86958-0c17-4485-956c-85f5f6e5712f None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:52.970467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86958-0c17-4485-956c-85f5f6e5712f None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:52.970747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86958-0c17-4485-956c-85f5f6e5712f None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:52.971552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc86958-0c17-4485-956c-85f5f6e5712f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:52.971870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc86958-0c17-4485-956c-85f5f6e5712f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:52.973478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc86958-0c17-4485-956c-85f5f6e5712f None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:52.988138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 435/882] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:52.993465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c742d1-4c7f-4e9c-8b8f-b65927f883a8 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:53.015853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:53.113025 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c742d1-4c7f-4e9c-8b8f-b65927f883a8 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:53.150949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b62912-321a-461c-b31c-857ca7b45330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:53.152495 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b62912-321a-461c-b31c-857ca7b45330 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:53.152936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b62912-321a-461c-b31c-857ca7b45330 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:53.153374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b62912-321a-461c-b31c-857ca7b45330 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:53.162231 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b62912-321a-461c-b31c-857ca7b45330 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:53.163939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b62912-321a-461c-b31c-857ca7b45330 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-03b62912-321a-461c-b31c-857ca7b45330 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:53.180366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:53.181693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:53.182922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:53.182922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:53.182922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:53.182922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:53.182922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:53.182922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:53.182922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:53.182922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:53.182922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:46:53.182922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:53.185268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 448/883] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 12:46:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:53.190736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc86958-0c17-4485-956c-85f5f6e5712f None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:53.295127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc86958-0c17-4485-956c-85f5f6e5712f None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:53.319264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:53.336022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee2d1d-4cf5-4ede-beba-85093b030643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:53.336983 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee2d1d-4cf5-4ede-beba-85093b030643 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:53.337424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee2d1d-4cf5-4ede-beba-85093b030643 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:53.337854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee2d1d-4cf5-4ede-beba-85093b030643 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:53.338865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee2d1d-4cf5-4ede-beba-85093b030643 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:53.339318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee2d1d-4cf5-4ede-beba-85093b030643 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:53.340894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee2d1d-4cf5-4ede-beba-85093b030643 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:53.360964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 436/884] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 12:46:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:53.366504 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b62912-321a-461c-b31c-857ca7b45330 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:53.466705 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:53.498688 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b62912-321a-461c-b31c-857ca7b45330 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:53.539032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336c6cc-1640-48b9-9182-898c6f3ea616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:53.539032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336c6cc-1640-48b9-9182-898c6f3ea616 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:53.539032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336c6cc-1640-48b9-9182-898c6f3ea616 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:53.539032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336c6cc-1640-48b9-9182-898c6f3ea616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:53.610949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336c6cc-1640-48b9-9182-898c6f3ea616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:53.612521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336c6cc-1640-48b9-9182-898c6f3ea616 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:53.673867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb11942-50f7-4b4f-9190-f9c1ac6ce018 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:53.674548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb11942-50f7-4b4f-9190-f9c1ac6ce018 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:53.675026 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb11942-50f7-4b4f-9190-f9c1ac6ce018 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:53.680302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 437/885] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:46:53.682052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2f1f1-2881-4e78-9f8d-8794580f7f85 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:53.682589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2f1f1-2881-4e78-9f8d-8794580f7f85 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:53.683062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2f1f1-2881-4e78-9f8d-8794580f7f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:53.700440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:53.705247 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e5d2f1f1-2881-4e78-9f8d-8794580f7f85 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:53.706709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e5d2f1f1-2881-4e78-9f8d-8794580f7f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:53.726472 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 449/886] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:46:53 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:46:53.732009 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336c6cc-1640-48b9-9182-898c6f3ea616 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:53.802399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 438/887] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:53.822172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2f1f1-2881-4e78-9f8d-8794580f7f85 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:53.843870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336c6cc-1640-48b9-9182-898c6f3ea616 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:53.882668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f1a4a-6662-4cbe-8169-c8b8a063fac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:53.882668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f1a4a-6662-4cbe-8169-c8b8a063fac8 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:53.882668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f1a4a-6662-4cbe-8169-c8b8a063fac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:53.882668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f1a4a-6662-4cbe-8169-c8b8a063fac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:53.895248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f1a4a-6662-4cbe-8169-c8b8a063fac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:53.897351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f1a4a-6662-4cbe-8169-c8b8a063fac8 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-ff7f1a4a-6662-4cbe-8169-c8b8a063fac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:53.920833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:53.921804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:53.923323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:53.923323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:53.923323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:53.923323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:53.923834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 450/888] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:53.925216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd986-dfd6-4256-8e3b-f931824a733d None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:53.925567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd986-dfd6-4256-8e3b-f931824a733d None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:53.925923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd986-dfd6-4256-8e3b-f931824a733d None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:53.927493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 451/889] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:53.928286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd986-dfd6-4256-8e3b-f931824a733d None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:53.949661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2f1f1-2881-4e78-9f8d-8794580f7f85 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:53.961905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:53.963382 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:53.992464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b98fc1-adde-4e32-b3b3-160de9056593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:53.993196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b98fc1-adde-4e32-b3b3-160de9056593 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:53.993474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b98fc1-adde-4e32-b3b3-160de9056593 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:53.993835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b98fc1-adde-4e32-b3b3-160de9056593 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:53.994431 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b98fc1-adde-4e32-b3b3-160de9056593 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:53.995792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b98fc1-adde-4e32-b3b3-160de9056593 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-b8b98fc1-adde-4e32-b3b3-160de9056593 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:54.008327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:54.009899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:54.013384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 439/890] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:54.015507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5730-f347-4198-9872-4f6e3db28d55 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:54.015507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5730-f347-4198-9872-4f6e3db28d55 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:54.015507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5730-f347-4198-9872-4f6e3db28d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:54.043589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-c56d5730-f347-4198-9872-4f6e3db28d55 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:54.045555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c56d5730-f347-4198-9872-4f6e3db28d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:54.066597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd986-dfd6-4256-8e3b-f931824a733d None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:54.115512 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1273a-5835-4049-84a8-10e0b893bf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:54.116658 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1273a-5835-4049-84a8-10e0b893bf93 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:54.116968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1273a-5835-4049-84a8-10e0b893bf93 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:54.118080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1273a-5835-4049-84a8-10e0b893bf93 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:54.118961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d1273a-5835-4049-84a8-10e0b893bf93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:54.119270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d1273a-5835-4049-84a8-10e0b893bf93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:54.120664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d1273a-5835-4049-84a8-10e0b893bf93 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:54.128028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 440/891] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:54.128677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d5730-f347-4198-9872-4f6e3db28d55 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:54.161627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 452/892] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:54.162455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1273a-5835-4049-84a8-10e0b893bf93 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:54.247802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d5730-f347-4198-9872-4f6e3db28d55 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:54.249816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:54.276206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:54.290616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6ffc7-cb52-4c1e-aafa-084b376e5996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:54.290844 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1273a-5835-4049-84a8-10e0b893bf93 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:54.291302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6ffc7-cb52-4c1e-aafa-084b376e5996 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:54.291594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6ffc7-cb52-4c1e-aafa-084b376e5996 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:54.291876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6ffc7-cb52-4c1e-aafa-084b376e5996 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:54.292600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6ffc7-cb52-4c1e-aafa-084b376e5996 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:54.292911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6ffc7-cb52-4c1e-aafa-084b376e5996 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:54.294341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6ffc7-cb52-4c1e-aafa-084b376e5996 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:54.299845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 441/893] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:54.299845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6ffc7-cb52-4c1e-aafa-084b376e5996 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:54.332668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049d3b3-91da-4fc9-bedd-457008d4e8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:54.333489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049d3b3-91da-4fc9-bedd-457008d4e8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:54.333768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049d3b3-91da-4fc9-bedd-457008d4e8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:54.334057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049d3b3-91da-4fc9-bedd-457008d4e8f1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:54.334743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049d3b3-91da-4fc9-bedd-457008d4e8f1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:54.336126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049d3b3-91da-4fc9-bedd-457008d4e8f1 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-8049d3b3-91da-4fc9-bedd-457008d4e8f1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:54.350533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:54.356836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:54.357999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 453/894] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 12:46:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:54.359010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049d3b3-91da-4fc9-bedd-457008d4e8f1 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:54.431943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6ffc7-cb52-4c1e-aafa-084b376e5996 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:54.468200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b3f78-7f53-45ad-bde7-848c9dabbe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:54.468999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3f78-7f53-45ad-bde7-848c9dabbe12 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:54.469288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3f78-7f53-45ad-bde7-848c9dabbe12 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:54.469566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3f78-7f53-45ad-bde7-848c9dabbe12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:54.476436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b3f78-7f53-45ad-bde7-848c9dabbe12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:54.477878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b3f78-7f53-45ad-bde7-848c9dabbe12 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:54.479498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049d3b3-91da-4fc9-bedd-457008d4e8f1 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-c67b3f78-7f53-45ad-bde7-848c9dabbe12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:54.492095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:54.493203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:54.494416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:54.494416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:54.494416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:54.494416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:54.494416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:54.494416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:54.494416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:54.494416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:54.495594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 442/895] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:54.496387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b3f78-7f53-45ad-bde7-848c9dabbe12 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:54.520147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd51770-ff35-4f56-ac0d-9065a048da39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:54.520737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51770-ff35-4f56-ac0d-9065a048da39 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:54.521022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51770-ff35-4f56-ac0d-9065a048da39 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:54.521299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51770-ff35-4f56-ac0d-9065a048da39 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:54.521997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd51770-ff35-4f56-ac0d-9065a048da39 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:54.522344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd51770-ff35-4f56-ac0d-9065a048da39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:54.523698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd51770-ff35-4f56-ac0d-9065a048da39 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:54.528372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:54.528724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 454/896] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:54.529492 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd51770-ff35-4f56-ac0d-9065a048da39 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:54.551459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:54.611800 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b3f78-7f53-45ad-bde7-848c9dabbe12 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:54.639208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd51770-ff35-4f56-ac0d-9065a048da39 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:54.648053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3582c95-2bcf-42e1-bafb-e2e62d992202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:54.648807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3582c95-2bcf-42e1-bafb-e2e62d992202 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:54.649172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3582c95-2bcf-42e1-bafb-e2e62d992202 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:54.649517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3582c95-2bcf-42e1-bafb-e2e62d992202 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:54.650312 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3582c95-2bcf-42e1-bafb-e2e62d992202 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:54.650714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3582c95-2bcf-42e1-bafb-e2e62d992202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:54.652139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3582c95-2bcf-42e1-bafb-e2e62d992202 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:54.661277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 443/897] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 12:46:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:54.661277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3582c95-2bcf-42e1-bafb-e2e62d992202 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:54.680415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66348f2c-cc36-491f-b7f5-1f5fc068b237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:54.681086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66348f2c-cc36-491f-b7f5-1f5fc068b237 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:54.681373 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66348f2c-cc36-491f-b7f5-1f5fc068b237 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:54.681652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66348f2c-cc36-491f-b7f5-1f5fc068b237 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:54.682478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66348f2c-cc36-491f-b7f5-1f5fc068b237 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:54.682796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66348f2c-cc36-491f-b7f5-1f5fc068b237 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:54.684234 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66348f2c-cc36-491f-b7f5-1f5fc068b237 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:54.702205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 455/898] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:54.702205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66348f2c-cc36-491f-b7f5-1f5fc068b237 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:54.756569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:54.767328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3582c95-2bcf-42e1-bafb-e2e62d992202 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:54.791724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:54.803010 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0eb8f-463b-45a4-acb2-8c5e99bda2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:54.803673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0eb8f-463b-45a4-acb2-8c5e99bda2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:54.803953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0eb8f-463b-45a4-acb2-8c5e99bda2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:54.804226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0eb8f-463b-45a4-acb2-8c5e99bda2b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:54.805020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0eb8f-463b-45a4-acb2-8c5e99bda2b4 None admin] RBAC: Adding query filter params (interface=public, service_id=1cc75d2e24224b9db995ae7d313e38e6, region_id=RegionOne) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:54.805329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0eb8f-463b-45a4-acb2-8c5e99bda2b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:54.806714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0eb8f-463b-45a4-acb2-8c5e99bda2b4 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:54.811748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 444/899] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 12:46:54 2022] GET /identity/v3/endpoints?service_id=1cc75d2e24224b9db995ae7d313e38e6&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:54.812566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0eb8f-463b-45a4-acb2-8c5e99bda2b4 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:54.824782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66348f2c-cc36-491f-b7f5-1f5fc068b237 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:54.864524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76088887-5937-4cf3-96c3-6dc48a561f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:54.865200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76088887-5937-4cf3-96c3-6dc48a561f7a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:54.865490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76088887-5937-4cf3-96c3-6dc48a561f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:54.865770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76088887-5937-4cf3-96c3-6dc48a561f7a None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:54.866511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76088887-5937-4cf3-96c3-6dc48a561f7a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:54.866823 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76088887-5937-4cf3-96c3-6dc48a561f7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:54.868198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76088887-5937-4cf3-96c3-6dc48a561f7a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:54.882356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 456/900] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 12:46:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:54.882356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76088887-5937-4cf3-96c3-6dc48a561f7a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:54.934748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0eb8f-463b-45a4-acb2-8c5e99bda2b4 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:54.977248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4947f76-c562-4811-8d35-9141503e8d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:54.977954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4947f76-c562-4811-8d35-9141503e8d7b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:54.978246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4947f76-c562-4811-8d35-9141503e8d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:54.978579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4947f76-c562-4811-8d35-9141503e8d7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:54.979428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4947f76-c562-4811-8d35-9141503e8d7b None admin] RBAC: Adding query filter params (role.id=b138ce6a3b514198ba4a9bdddb99ab81, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=122fdcf8f13348cf858555a565e5cf09) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:54.979743 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4947f76-c562-4811-8d35-9141503e8d7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:54.981202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4947f76-c562-4811-8d35-9141503e8d7b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:54.994733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 445/901] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:46:54 2022] GET /identity/v3/role_assignments?user.id=122fdcf8f13348cf858555a565e5cf09&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=b138ce6a3b514198ba4a9bdddb99ab81 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:55.017744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76088887-5937-4cf3-96c3-6dc48a561f7a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:55.065048 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8e7b3-9fe8-426e-ae75-927193cde982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:55.066018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8e7b3-9fe8-426e-ae75-927193cde982 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:55.066511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8e7b3-9fe8-426e-ae75-927193cde982 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:55.066954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8e7b3-9fe8-426e-ae75-927193cde982 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:55.067837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8e7b3-9fe8-426e-ae75-927193cde982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:55.069443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8e7b3-9fe8-426e-ae75-927193cde982 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:55.077675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 457/902] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 12:46:54 2022] GET /identity/v3/services => generated 1268 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:46:55.333547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eda85-ca2f-4378-bbdc-4aaee8102d5e None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:55.334047 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eda85-ca2f-4378-bbdc-4aaee8102d5e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:55.334047 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eda85-ca2f-4378-bbdc-4aaee8102d5e None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:55.335642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 446/903] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:55.341016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85489c7c-cadf-4b39-8514-6864032621eb None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:55.341421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85489c7c-cadf-4b39-8514-6864032621eb None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:55.341564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85489c7c-cadf-4b39-8514-6864032621eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:55.353357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:55.360532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-85489c7c-cadf-4b39-8514-6864032621eb None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:55.362309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-85489c7c-cadf-4b39-8514-6864032621eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:55.442701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 458/904] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:55.465206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0eda85-ca2f-4378-bbdc-4aaee8102d5e None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:55.486744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:55.576449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0eda85-ca2f-4378-bbdc-4aaee8102d5e None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:55.613979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e6f55-f03e-4699-8a6d-545731e0824f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:55.614820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e6f55-f03e-4699-8a6d-545731e0824f None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:55.615109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e6f55-f03e-4699-8a6d-545731e0824f None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:55.615385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e6f55-f03e-4699-8a6d-545731e0824f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:55.616043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e6f55-f03e-4699-8a6d-545731e0824f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:55.617410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e6f55-f03e-4699-8a6d-545731e0824f None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-7c0e6f55-f03e-4699-8a6d-545731e0824f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:55.630802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:55.631788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:55.632644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 447/905] 10.222.0.45 () {56 vars in 1132 bytes} [Tue May 24 12:46:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:55.638619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85489c7c-cadf-4b39-8514-6864032621eb None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:55.740085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85489c7c-cadf-4b39-8514-6864032621eb None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:55.776669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986b63a-2930-47e5-afd7-53da7487e67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:55.777335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986b63a-2930-47e5-afd7-53da7487e67f None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:55.777622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986b63a-2930-47e5-afd7-53da7487e67f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:55.777900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986b63a-2930-47e5-afd7-53da7487e67f None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:55.778639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986b63a-2930-47e5-afd7-53da7487e67f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:55.778948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986b63a-2930-47e5-afd7-53da7487e67f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:55.780898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986b63a-2930-47e5-afd7-53da7487e67f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:55.782846 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:55.786392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 459/906] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 12:46:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:55.792351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e6f55-f03e-4699-8a6d-545731e0824f None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:55.884729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:55.903721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e6f55-f03e-4699-8a6d-545731e0824f None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:55.937435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a1ee7-a51b-4047-a552-34672918081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:55.938529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a1ee7-a51b-4047-a552-34672918081e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:55.938813 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a1ee7-a51b-4047-a552-34672918081e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:55.939085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a1ee7-a51b-4047-a552-34672918081e None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:55.939787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a1ee7-a51b-4047-a552-34672918081e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:55.940091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a1ee7-a51b-4047-a552-34672918081e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:55.941452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a1ee7-a51b-4047-a552-34672918081e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:55.946688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 448/907] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 12:46:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:55.952371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986b63a-2930-47e5-afd7-53da7487e67f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:56.052857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986b63a-2930-47e5-afd7-53da7487e67f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:56.090709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da412fd6-928f-4f51-85c1-f24a86ebc116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:56.091372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da412fd6-928f-4f51-85c1-f24a86ebc116 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:56.091650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da412fd6-928f-4f51-85c1-f24a86ebc116 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:56.091925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da412fd6-928f-4f51-85c1-f24a86ebc116 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:56.092736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da412fd6-928f-4f51-85c1-f24a86ebc116 None admin] RBAC: Adding query filter params (interface=public, service_id=665371d31ef74940bf89d1056766e958, region_id=RegionOne) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:56.093043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da412fd6-928f-4f51-85c1-f24a86ebc116 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:56.094426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da412fd6-928f-4f51-85c1-f24a86ebc116 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:56.099996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 460/908] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 12:46:55 2022] GET /identity/v3/endpoints?service_id=665371d31ef74940bf89d1056766e958&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:56.105328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a1ee7-a51b-4047-a552-34672918081e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:56.210883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a1ee7-a51b-4047-a552-34672918081e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:56.245516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75defb8-57ca-4ab8-b33e-137709b71dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:56.246196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75defb8-57ca-4ab8-b33e-137709b71dfc None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:56.246500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75defb8-57ca-4ab8-b33e-137709b71dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:56.246775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75defb8-57ca-4ab8-b33e-137709b71dfc None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:56.247463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75defb8-57ca-4ab8-b33e-137709b71dfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:56.248932 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75defb8-57ca-4ab8-b33e-137709b71dfc None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:56.255979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 449/909] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 12:46:56 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:46:57.180072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc957779-f2d2-48f6-bfc8-4a872b0b2915 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:57.180598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc957779-f2d2-48f6-bfc8-4a872b0b2915 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:57.180598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc957779-f2d2-48f6-bfc8-4a872b0b2915 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:57.182334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 461/910] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:57.187913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acbd1b-e203-4824-a697-27d3748b24bd None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:57.188316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acbd1b-e203-4824-a697-27d3748b24bd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:57.188515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acbd1b-e203-4824-a697-27d3748b24bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:57.205942 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-80acbd1b-e203-4824-a697-27d3748b24bd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:57.207189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-80acbd1b-e203-4824-a697-27d3748b24bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:57.210493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:57.284748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 450/911] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:57.306989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc957779-f2d2-48f6-bfc8-4a872b0b2915 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:57.318164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77c952-2fc8-49b4-bbd0-1334ca9200bb None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:57.318846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77c952-2fc8-49b4-bbd0-1334ca9200bb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:57.319136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77c952-2fc8-49b4-bbd0-1334ca9200bb None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:57.321368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 451/912] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:46:57.328206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b28ee0-4493-43d9-aa0c-ba3c69eb0c5c None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:57.328494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b28ee0-4493-43d9-aa0c-ba3c69eb0c5c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:57.328774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b28ee0-4493-43d9-aa0c-ba3c69eb0c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:57.350949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-20b28ee0-4493-43d9-aa0c-ba3c69eb0c5c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:57.352214 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-20b28ee0-4493-43d9-aa0c-ba3c69eb0c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:57.375873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:57.424815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc957779-f2d2-48f6-bfc8-4a872b0b2915 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:57.445050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 452/913] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:46:57.448460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738ffff-2246-4235-9072-d1be3aba42eb None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:57.448960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738ffff-2246-4235-9072-d1be3aba42eb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:57.449676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738ffff-2246-4235-9072-d1be3aba42eb None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:57.451243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 453/914] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:57.458115 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd72f5-2ad8-426d-a860-4256046b20f8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:57.458449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd72f5-2ad8-426d-a860-4256046b20f8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:57.458724 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd72f5-2ad8-426d-a860-4256046b20f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:57.467594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f26ea-4b70-411c-879c-3b63d95a11fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:57.468273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f26ea-4b70-411c-879c-3b63d95a11fc None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:57.468555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f26ea-4b70-411c-879c-3b63d95a11fc None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:57.468826 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f26ea-4b70-411c-879c-3b63d95a11fc None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:57.469564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f26ea-4b70-411c-879c-3b63d95a11fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:57.471430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f26ea-4b70-411c-879c-3b63d95a11fc None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:57.476247 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-2bdd72f5-2ad8-426d-a860-4256046b20f8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:57.477636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2bdd72f5-2ad8-426d-a860-4256046b20f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:57.494318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 462/915] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:46:57 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:46:57.494318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f26ea-4b70-411c-879c-3b63d95a11fc None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:57.561317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:57.574462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 454/916] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:46:57.607065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd72f5-2ad8-426d-a860-4256046b20f8 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:57.656791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f26ea-4b70-411c-879c-3b63d95a11fc None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:57.687429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:57.705398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955d5aa-80d0-42e4-bb32-95f25338f2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:57.706038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d5aa-80d0-42e4-bb32-95f25338f2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:57.706355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d5aa-80d0-42e4-bb32-95f25338f2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:57.706636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d5aa-80d0-42e4-bb32-95f25338f2d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:57.720427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955d5aa-80d0-42e4-bb32-95f25338f2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:46:57.726741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd72f5-2ad8-426d-a860-4256046b20f8 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-f955d5aa-80d0-42e4-bb32-95f25338f2d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:46:57.728569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:57.729465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:57.730559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 463/917] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 12:46:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:57.735563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955d5aa-80d0-42e4-bb32-95f25338f2d3 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:57.767038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96394207-deef-42b7-ae74-c3b249951354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:57.768182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96394207-deef-42b7-ae74-c3b249951354 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:57.768763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96394207-deef-42b7-ae74-c3b249951354 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:57.769249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96394207-deef-42b7-ae74-c3b249951354 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:57.770124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96394207-deef-42b7-ae74-c3b249951354 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:57.771864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96394207-deef-42b7-ae74-c3b249951354 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-96394207-deef-42b7-ae74-c3b249951354 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:57.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:57.785900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 12:46:57.786794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:57.789828 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 455/918] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:57.796772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96394207-deef-42b7-ae74-c3b249951354 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:57.866897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:57.871721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955d5aa-80d0-42e4-bb32-95f25338f2d3 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:57.914928 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96394207-deef-42b7-ae74-c3b249951354 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:57.918946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd67f6-9ef9-4797-8891-4b1b5edf19be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:57.923070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd67f6-9ef9-4797-8891-4b1b5edf19be None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:57.923397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd67f6-9ef9-4797-8891-4b1b5edf19be None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:57.924217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd67f6-9ef9-4797-8891-4b1b5edf19be None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:57.924987 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd67f6-9ef9-4797-8891-4b1b5edf19be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:57.925301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd67f6-9ef9-4797-8891-4b1b5edf19be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:57.927182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd67f6-9ef9-4797-8891-4b1b5edf19be None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:57.939667 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 464/919] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 12:46:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:57.947516 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd67f6-9ef9-4797-8891-4b1b5edf19be None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:57.957198 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c203886-e20f-4431-95cf-b419c501fcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:57.958215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c203886-e20f-4431-95cf-b419c501fcec None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:57.958774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c203886-e20f-4431-95cf-b419c501fcec None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:57.959310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c203886-e20f-4431-95cf-b419c501fcec None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:57.960376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c203886-e20f-4431-95cf-b419c501fcec None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:57.960883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c203886-e20f-4431-95cf-b419c501fcec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:57.962573 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c203886-e20f-4431-95cf-b419c501fcec None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:57.972815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 456/920] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:57.974342 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c203886-e20f-4431-95cf-b419c501fcec None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:57.999549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:58.063157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd67f6-9ef9-4797-8891-4b1b5edf19be None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:58.084575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c203886-e20f-4431-95cf-b419c501fcec None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:58.108629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482512c-a80c-46f3-86e4-ba6ed3ae698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:58.110159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482512c-a80c-46f3-86e4-ba6ed3ae698e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:58.110853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482512c-a80c-46f3-86e4-ba6ed3ae698e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:58.111307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482512c-a80c-46f3-86e4-ba6ed3ae698e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:58.113260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:58.127775 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482512c-a80c-46f3-86e4-ba6ed3ae698e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:58.127969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de87c93-7b1f-47c6-a2b3-a015399bb4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:58.128552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de87c93-7b1f-47c6-a2b3-a015399bb4cc None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:58.128846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de87c93-7b1f-47c6-a2b3-a015399bb4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:58.129126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de87c93-7b1f-47c6-a2b3-a015399bb4cc None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:58.130057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482512c-a80c-46f3-86e4-ba6ed3ae698e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:58.131443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de87c93-7b1f-47c6-a2b3-a015399bb4cc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:58.131761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de87c93-7b1f-47c6-a2b3-a015399bb4cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:58.133644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de87c93-7b1f-47c6-a2b3-a015399bb4cc None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:58.144130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 457/921] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 12:46:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:58.145644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de87c93-7b1f-47c6-a2b3-a015399bb4cc None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-5482512c-a80c-46f3-86e4-ba6ed3ae698e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:58.152761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:58.153650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:58.155763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:58.155763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:58.155763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:58.155763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:58.155763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 465/922] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:58.159939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482512c-a80c-46f3-86e4-ba6ed3ae698e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:58.274192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de87c93-7b1f-47c6-a2b3-a015399bb4cc None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:58.281932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:58.286439 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482512c-a80c-46f3-86e4-ba6ed3ae698e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:58.322497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ed29e-9662-4ba6-b769-5775531ac4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:58.323093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ed29e-9662-4ba6-b769-5775531ac4b1 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:58.323381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ed29e-9662-4ba6-b769-5775531ac4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:58.323660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ed29e-9662-4ba6-b769-5775531ac4b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:58.324402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ed29e-9662-4ba6-b769-5775531ac4b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:58.325676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93304df-386f-435d-a2f7-770b95376b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:58.325809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ed29e-9662-4ba6-b769-5775531ac4b1 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:58.326946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93304df-386f-435d-a2f7-770b95376b0c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:58.327427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93304df-386f-435d-a2f7-770b95376b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:58.328080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93304df-386f-435d-a2f7-770b95376b0c None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:58.328992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93304df-386f-435d-a2f7-770b95376b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:58.329502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93304df-386f-435d-a2f7-770b95376b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:58.331099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93304df-386f-435d-a2f7-770b95376b0c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:58.344058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 466/923] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:58.350469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93304df-386f-435d-a2f7-770b95376b0c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:58.361570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 458/924] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 12:46:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:46:58.486806 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93304df-386f-435d-a2f7-770b95376b0c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:58.531255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b5ed2-1021-4dbd-8a52-6ebe2c8f628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:58.531989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b5ed2-1021-4dbd-8a52-6ebe2c8f628c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:58.532270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b5ed2-1021-4dbd-8a52-6ebe2c8f628c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:58.532542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b5ed2-1021-4dbd-8a52-6ebe2c8f628c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:58.541201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b5ed2-1021-4dbd-8a52-6ebe2c8f628c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:58.542767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b5ed2-1021-4dbd-8a52-6ebe2c8f628c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-bc9b5ed2-1021-4dbd-8a52-6ebe2c8f628c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:46:58.558707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:58.559679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:58.560616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:58.560616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:58.560616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:46:58.560616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:46:58.560616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:46:58.560616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:46:58.560616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:46:58.560616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:46:58.560616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:46:58.560616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:58.560616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 467/925] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 12:46:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:58.567492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ed29e-9662-4ba6-b769-5775531ac4b1 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:58.582243 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e625df-7c30-4763-984a-ad3f00ffaf53 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:58.582570 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e625df-7c30-4763-984a-ad3f00ffaf53 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:58.582842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e625df-7c30-4763-984a-ad3f00ffaf53 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:58.584373 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 468/926] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:58.590575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857a72e-2d1b-422a-93e0-c0b0a1c926a3 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:58.590862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857a72e-2d1b-422a-93e0-c0b0a1c926a3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:58.591140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857a72e-2d1b-422a-93e0-c0b0a1c926a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:58.612003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9857a72e-2d1b-422a-93e0-c0b0a1c926a3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:58.613201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9857a72e-2d1b-422a-93e0-c0b0a1c926a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:58.623957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:58.642649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:58.687616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 469/927] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:46:58.708407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ed29e-9662-4ba6-b769-5775531ac4b1 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:58.715113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857a72e-2d1b-422a-93e0-c0b0a1c926a3 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:58.757271 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9105c-5a70-4e67-a341-3b22be117dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:58.757953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9105c-5a70-4e67-a341-3b22be117dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:58.758819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9105c-5a70-4e67-a341-3b22be117dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:58.759103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9105c-5a70-4e67-a341-3b22be117dc9 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:58.759945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9105c-5a70-4e67-a341-3b22be117dc9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:58.760969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9105c-5a70-4e67-a341-3b22be117dc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:58.762905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9105c-5a70-4e67-a341-3b22be117dc9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:58.793100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 459/928] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 12:46:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:58.794831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9105c-5a70-4e67-a341-3b22be117dc9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:58.830073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857a72e-2d1b-422a-93e0-c0b0a1c926a3 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:58.866617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2cc97-291b-4da3-8914-45c52baca2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:58.867540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2cc97-291b-4da3-8914-45c52baca2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:58.867881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2cc97-291b-4da3-8914-45c52baca2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:58.868223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2cc97-291b-4da3-8914-45c52baca2d9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:58.868988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2cc97-291b-4da3-8914-45c52baca2d9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:58.870457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2cc97-291b-4da3-8914-45c52baca2d9 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-3ac2cc97-291b-4da3-8914-45c52baca2d9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:58.885363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:58.886367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 12:46:58.887228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:46:58.891809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 470/929] 10.222.0.45 () {56 vars in 1132 bytes} [Tue May 24 12:46:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:46:58.893205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2cc97-291b-4da3-8914-45c52baca2d9 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:58.913765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:58.924010 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9105c-5a70-4e67-a341-3b22be117dc9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:58.926124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:58.963921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5816d-0baa-47fb-9713-7b38dec5de85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:58.965211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5816d-0baa-47fb-9713-7b38dec5de85 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:58.965562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5816d-0baa-47fb-9713-7b38dec5de85 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:58.965917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5816d-0baa-47fb-9713-7b38dec5de85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:58.977339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5816d-0baa-47fb-9713-7b38dec5de85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:58.978939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5816d-0baa-47fb-9713-7b38dec5de85 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-65d5816d-0baa-47fb-9713-7b38dec5de85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:46:59.005229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:59.006180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:59.007117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:46:59.007117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:46:59.007117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:46:59.007117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:59.011807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 460/930] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:59.012718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5816d-0baa-47fb-9713-7b38dec5de85 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:59.024460 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2cc97-291b-4da3-8914-45c52baca2d9 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:59.066773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f3a2e-17b4-4224-be92-66e059d1f4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:59.067554 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f3a2e-17b4-4224-be92-66e059d1f4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:59.067852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f3a2e-17b4-4224-be92-66e059d1f4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:59.068137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f3a2e-17b4-4224-be92-66e059d1f4c2 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:59.068851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f3a2e-17b4-4224-be92-66e059d1f4c2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:59.069166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f3a2e-17b4-4224-be92-66e059d1f4c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:59.070685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f3a2e-17b4-4224-be92-66e059d1f4c2 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:59.076000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 471/931] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 12:46:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:59.081484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f3a2e-17b4-4224-be92-66e059d1f4c2 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:59.134173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5816d-0baa-47fb-9713-7b38dec5de85 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:59.154688 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:59.173870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249ebba-b875-4eae-92f0-63d8d32428f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:59.175306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249ebba-b875-4eae-92f0-63d8d32428f5 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:59.175803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249ebba-b875-4eae-92f0-63d8d32428f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:59.176387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249ebba-b875-4eae-92f0-63d8d32428f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:59.177327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249ebba-b875-4eae-92f0-63d8d32428f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:59.177880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249ebba-b875-4eae-92f0-63d8d32428f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:59.179586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249ebba-b875-4eae-92f0-63d8d32428f5 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:59.188153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f3a2e-17b4-4224-be92-66e059d1f4c2 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:59.197193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 461/932] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:59.198528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249ebba-b875-4eae-92f0-63d8d32428f5 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:59.216079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:59.234065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e7bc7-b82f-426c-ad69-9bc086895dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:59.234922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e7bc7-b82f-426c-ad69-9bc086895dcc None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:59.235211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e7bc7-b82f-426c-ad69-9bc086895dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:59.235488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e7bc7-b82f-426c-ad69-9bc086895dcc None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:59.236505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e7bc7-b82f-426c-ad69-9bc086895dcc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:59.236828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e7bc7-b82f-426c-ad69-9bc086895dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:59.238474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e7bc7-b82f-426c-ad69-9bc086895dcc None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:59.246369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 472/933] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 12:46:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:46:59.251236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e7bc7-b82f-426c-ad69-9bc086895dcc None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:59.325476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249ebba-b875-4eae-92f0-63d8d32428f5 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:59.364907 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b42a21-f66c-42be-8230-b84fdb75263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:59.365409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e7bc7-b82f-426c-ad69-9bc086895dcc None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:59.366702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b42a21-f66c-42be-8230-b84fdb75263b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:59.367191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b42a21-f66c-42be-8230-b84fdb75263b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:59.367672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b42a21-f66c-42be-8230-b84fdb75263b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:59.375073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b42a21-f66c-42be-8230-b84fdb75263b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:59.376761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b42a21-f66c-42be-8230-b84fdb75263b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-c5b42a21-f66c-42be-8230-b84fdb75263b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:46:59.393225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:46:59.394152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:46:59.395404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:46:59.395404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:46:59.395404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:46:59.395404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:46:59.395404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:46:59.395404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:46:59.395404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:46:59.395404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:46:59.402381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 462/934] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:46:59.403152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b42a21-f66c-42be-8230-b84fdb75263b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:59.413673 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37484bf0-8a38-4a00-8a76-d588f143a98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:59.414553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37484bf0-8a38-4a00-8a76-d588f143a98b None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:59.414932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37484bf0-8a38-4a00-8a76-d588f143a98b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:59.415320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37484bf0-8a38-4a00-8a76-d588f143a98b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:59.416171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37484bf0-8a38-4a00-8a76-d588f143a98b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:59.417701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37484bf0-8a38-4a00-8a76-d588f143a98b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:59.422606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:59.442799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 473/935] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 12:46:59 2022] POST /identity/v3/endpoints => generated 347 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:46:59.507412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:46:59.537476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b42a21-f66c-42be-8230-b84fdb75263b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:59.584310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e173bf-e9dc-4ca9-8591-4bdaeb717cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:59.585343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e173bf-e9dc-4ca9-8591-4bdaeb717cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:59.585833 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e173bf-e9dc-4ca9-8591-4bdaeb717cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:59.586377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e173bf-e9dc-4ca9-8591-4bdaeb717cc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:59.587383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e173bf-e9dc-4ca9-8591-4bdaeb717cc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:46:59.587919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e173bf-e9dc-4ca9-8591-4bdaeb717cc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:59.589593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e173bf-e9dc-4ca9-8591-4bdaeb717cc4 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:59.604019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 463/936] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:46:59.611458 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37484bf0-8a38-4a00-8a76-d588f143a98b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:46:59.726638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37484bf0-8a38-4a00-8a76-d588f143a98b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:46:59.761779 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871af58-21c9-48b1-bc0a-a736cb1cd201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:46:59.762697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871af58-21c9-48b1-bc0a-a736cb1cd201 None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:59.762978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871af58-21c9-48b1-bc0a-a736cb1cd201 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:59.763252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871af58-21c9-48b1-bc0a-a736cb1cd201 None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/122fdcf8f13348cf858555a565e5cf09/roles/b138ce6a3b514198ba4a9bdddb99ab81` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:59.765683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871af58-21c9-48b1-bc0a-a736cb1cd201 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=122fdcf8f13348cf858555a565e5cf09, role_id=b138ce6a3b514198ba4a9bdddb99ab81)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:46:59.767225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871af58-21c9-48b1-bc0a-a736cb1cd201 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:46:59.782832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 474/937] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:46:59 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/122fdcf8f13348cf858555a565e5cf09/roles/b138ce6a3b514198ba4a9bdddb99ab81 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:46:59.894186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a55f3-51c8-44a4-b646-d0c5d6a51e46 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:59.894945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a55f3-51c8-44a4-b646-d0c5d6a51e46 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:59.895451 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a55f3-51c8-44a4-b646-d0c5d6a51e46 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:59.897348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 464/938] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:46:59.903472 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560f86e-6116-45e0-a1a1-d40a7b58e903 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:46:59.904060 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560f86e-6116-45e0-a1a1-d40a7b58e903 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:46:59.904635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560f86e-6116-45e0-a1a1-d40a7b58e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:46:59.923812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-5560f86e-6116-45e0-a1a1-d40a7b58e903 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:46:59.925394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5560f86e-6116-45e0-a1a1-d40a7b58e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:46:59.977885 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:00.000165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 475/939] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:46:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:00.023767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a55f3-51c8-44a4-b646-d0c5d6a51e46 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:00.129939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a55f3-51c8-44a4-b646-d0c5d6a51e46 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:00.160165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:00.166375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc77b9-f57a-4262-b963-ee2dc504a2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:00.167252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc77b9-f57a-4262-b963-ee2dc504a2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:00.167540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc77b9-f57a-4262-b963-ee2dc504a2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:00.167819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc77b9-f57a-4262-b963-ee2dc504a2e9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:00.168488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc77b9-f57a-4262-b963-ee2dc504a2e9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:00.170786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc77b9-f57a-4262-b963-ee2dc504a2e9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-b2bc77b9-f57a-4262-b963-ee2dc504a2e9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:00.184637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:00.185745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:00.186647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 465/940] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 12:47:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:00.192521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560f86e-6116-45e0-a1a1-d40a7b58e903 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:00.322839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560f86e-6116-45e0-a1a1-d40a7b58e903 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:00.358813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba1b21-dee5-400a-a1c7-fc4652765e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:00.359489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba1b21-dee5-400a-a1c7-fc4652765e83 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:00.359772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba1b21-dee5-400a-a1c7-fc4652765e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:00.360046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba1b21-dee5-400a-a1c7-fc4652765e83 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:00.360764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba1b21-dee5-400a-a1c7-fc4652765e83 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:00.361071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba1b21-dee5-400a-a1c7-fc4652765e83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:00.362475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba1b21-dee5-400a-a1c7-fc4652765e83 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:00.367716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 476/941] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 12:47:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:00.373518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc77b9-f57a-4262-b963-ee2dc504a2e9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:00.487898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc77b9-f57a-4262-b963-ee2dc504a2e9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:00.525234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d4155-ddb7-4f28-8dd7-4349c7e9f909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:00.525998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d4155-ddb7-4f28-8dd7-4349c7e9f909 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:00.526341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d4155-ddb7-4f28-8dd7-4349c7e9f909 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:00.526598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d4155-ddb7-4f28-8dd7-4349c7e9f909 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:00.527374 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d4155-ddb7-4f28-8dd7-4349c7e9f909 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:00.527682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d4155-ddb7-4f28-8dd7-4349c7e9f909 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:00.529092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d4155-ddb7-4f28-8dd7-4349c7e9f909 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:00.535775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 466/942] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 12:47:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:00.541696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba1b21-dee5-400a-a1c7-fc4652765e83 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:00.602927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:00.657391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba1b21-dee5-400a-a1c7-fc4652765e83 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:00.691549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee5621-7ada-4844-a525-1a2dcf364f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:00.692270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee5621-7ada-4844-a525-1a2dcf364f0d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:00.692561 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee5621-7ada-4844-a525-1a2dcf364f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:00.692836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee5621-7ada-4844-a525-1a2dcf364f0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:00.693642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee5621-7ada-4844-a525-1a2dcf364f0d None admin] RBAC: Adding query filter params (interface=public, service_id=6efb98fbee1c4569a51726c5c163dce2, region_id=RegionOne) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:00.693956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee5621-7ada-4844-a525-1a2dcf364f0d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:00.695413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee5621-7ada-4844-a525-1a2dcf364f0d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:00.704003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6ccc1-bca1-4d22-b311-7f4c08f77615 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:00.704364 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6ccc1-bca1-4d22-b311-7f4c08f77615 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:00.704562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6ccc1-bca1-4d22-b311-7f4c08f77615 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:00.705150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 477/943] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 12:47:00 2022] GET /identity/v3/endpoints?service_id=6efb98fbee1c4569a51726c5c163dce2&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:00.706023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 467/944] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:00.710803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee5621-7ada-4844-a525-1a2dcf364f0d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:00.712191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad857725-9a73-4950-ac28-485e6f31a3a9 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:00.712582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad857725-9a73-4950-ac28-485e6f31a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:00.712957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad857725-9a73-4950-ac28-485e6f31a3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:00.729790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-ad857725-9a73-4950-ac28-485e6f31a3a9 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:00.731242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ad857725-9a73-4950-ac28-485e6f31a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:00.797698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:00.818378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 468/945] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:47:00.827916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee5621-7ada-4844-a525-1a2dcf364f0d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:00.839248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:00.842919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad857725-9a73-4950-ac28-485e6f31a3a9 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:00.871382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578d054-1ca3-484a-b4c6-f15efc57bd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:00.872115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578d054-1ca3-484a-b4c6-f15efc57bd30 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:00.872498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578d054-1ca3-484a-b4c6-f15efc57bd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:00.872779 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578d054-1ca3-484a-b4c6-f15efc57bd30 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:00.873682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578d054-1ca3-484a-b4c6-f15efc57bd30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:00.875156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578d054-1ca3-484a-b4c6-f15efc57bd30 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:00.882277 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 478/946] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 12:47:00 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:47:00.952057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad857725-9a73-4950-ac28-485e6f31a3a9 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:00.991834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71eb55-d8f4-4b08-95f9-70f238089660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:00.992521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71eb55-d8f4-4b08-95f9-70f238089660 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:00.992805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71eb55-d8f4-4b08-95f9-70f238089660 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:00.993077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71eb55-d8f4-4b08-95f9-70f238089660 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:00.993816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71eb55-d8f4-4b08-95f9-70f238089660 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:00.995321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71eb55-d8f4-4b08-95f9-70f238089660 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:00.996720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: WARNING keystone.server.flask.common [None req-0a71eb55-d8f4-4b08-95f9-70f238089660 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 24 12:47:01.017595 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.sql.core [None req-0a71eb55-d8f4-4b08-95f9-70f238089660 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 12:47:01.017595 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 24 12:47:01.017595 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [parameters: {'id': 'a18c15e4856a4ee5a0178a7af077798a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 24 12:47:01.017595 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80091) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-0a71eb55-d8f4-4b08-95f9-70f238089660 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a18c15e4856a4ee5a0178a7af077798a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a18c15e4856a4ee5a0178a7af077798a. May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application result = self._query(query) May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application conn.query(q) May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 12:47:01.035124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application result.read() May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 24 12:47:01.037090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return project_ref.to_dict() May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application next(self.gen) May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application yield resource May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application next(self.gen) May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self.session.rollback() May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self.force_reraise() May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise self.value May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application session.commit() May 24 12:47:01.039413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._prepare_impl() May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self.session.flush() May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._flush(objects) May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application compat.raise_( May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application flush_context.execute() May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 12:47:01.041560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rec.execute(self) May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application c = connection._execute_20( May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 12:47:01.043627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application result = self._query(query) May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application conn.query(q) May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application result.read() May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 12:47:01.044625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [parameters: {'id': 'a18c15e4856a4ee5a0178a7af077798a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:01.046602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:01.047415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:01.048370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:01.048370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 24 12:47:01.048370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 24 12:47:01.048370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:01.048370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:01.048370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 24 12:47:01.048370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.Conflict( May 24 12:47:01.048370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a18c15e4856a4ee5a0178a7af077798a. May 24 12:47:01.048370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:01.048370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 469/947] 10.222.0.45 () {60 vars in 1140 bytes} [Tue May 24 12:47:00 2022] POST /identity/v3/projects => generated 262 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 24 12:47:01.048935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578d054-1ca3-484a-b4c6-f15efc57bd30 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:01.157326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578d054-1ca3-484a-b4c6-f15efc57bd30 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:01.192645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35439a0-fc82-4abe-bcb0-fc42df375147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:01.193491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35439a0-fc82-4abe-bcb0-fc42df375147 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:01.193777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35439a0-fc82-4abe-bcb0-fc42df375147 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:01.194048 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35439a0-fc82-4abe-bcb0-fc42df375147 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:01.200796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35439a0-fc82-4abe-bcb0-fc42df375147 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:01.202308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35439a0-fc82-4abe-bcb0-fc42df375147 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-c35439a0-fc82-4abe-bcb0-fc42df375147 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:47:01.215794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:01.216740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:01.218205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:47:01.218205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:47:01.218205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:47:01.218205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:47:01.218205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:47:01.218205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:47:01.218205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:47:01.218205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:01.222195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 479/948] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:01.223816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71eb55-d8f4-4b08-95f9-70f238089660 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:01.272175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:01.329115 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71eb55-d8f4-4b08-95f9-70f238089660 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:01.363322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71150a-4c83-466e-9686-7b8436e9df18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:01.363993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71150a-4c83-466e-9686-7b8436e9df18 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:01.364274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71150a-4c83-466e-9686-7b8436e9df18 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:01.364550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71150a-4c83-466e-9686-7b8436e9df18 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:01.365303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71150a-4c83-466e-9686-7b8436e9df18 None admin] RBAC: Adding query filter params (name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:01.365607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71150a-4c83-466e-9686-7b8436e9df18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:01.367245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71150a-4c83-466e-9686-7b8436e9df18 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:01.390328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 470/949] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:47:01 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:01.854739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4f725-22da-4834-82b3-1666c9a72c70 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:01.855763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4f725-22da-4834-82b3-1666c9a72c70 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:01.856249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4f725-22da-4834-82b3-1666c9a72c70 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:01.858141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 480/950] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:01.863765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc91a6-002f-4a02-9208-74b0d61abb47 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:01.864282 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc91a6-002f-4a02-9208-74b0d61abb47 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:01.864757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc91a6-002f-4a02-9208-74b0d61abb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:01.883017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e2cc91a6-002f-4a02-9208-74b0d61abb47 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:01.884851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e2cc91a6-002f-4a02-9208-74b0d61abb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:01.961794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 471/951] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:01.984205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4f725-22da-4834-82b3-1666c9a72c70 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:02.027748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:02.097622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4f725-22da-4834-82b3-1666c9a72c70 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:02.116569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fe918-3986-4f67-b6c0-b1183db88da6 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:02.116846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fe918-3986-4f67-b6c0-b1183db88da6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:02.117066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fe918-3986-4f67-b6c0-b1183db88da6 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:02.118843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 472/952] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:47:02.124169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acf9e8-0500-4d13-bae8-51fd6cf0053e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:02.124467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acf9e8-0500-4d13-bae8-51fd6cf0053e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:02.124748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acf9e8-0500-4d13-bae8-51fd6cf0053e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:02.139983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dcaf0-a7e5-46b2-b97d-0c91d6747dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:02.141058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dcaf0-a7e5-46b2-b97d-0c91d6747dcf None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:02.141343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dcaf0-a7e5-46b2-b97d-0c91d6747dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:02.141618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dcaf0-a7e5-46b2-b97d-0c91d6747dcf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:02.142274 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2dcaf0-a7e5-46b2-b97d-0c91d6747dcf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:02.143729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2dcaf0-a7e5-46b2-b97d-0c91d6747dcf None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:02.144169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-d3acf9e8-0500-4d13-bae8-51fd6cf0053e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:02.145400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d3acf9e8-0500-4d13-bae8-51fd6cf0053e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-6e2dcaf0-a7e5-46b2-b97d-0c91d6747dcf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:02.159213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:02.160338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 12:47:02.161265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:02.166241 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 481/953] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:47:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:47:02.167584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dcaf0-a7e5-46b2-b97d-0c91d6747dcf None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:02.181702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:02.235025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 473/954] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:47:02.260560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acf9e8-0500-4d13-bae8-51fd6cf0053e None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:02.294351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:02.299334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dcaf0-a7e5-46b2-b97d-0c91d6747dcf None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:02.336567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61264dbe-ae23-46c9-b0fa-27886259ab37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:02.337292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61264dbe-ae23-46c9-b0fa-27886259ab37 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:02.337292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61264dbe-ae23-46c9-b0fa-27886259ab37 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:02.337292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61264dbe-ae23-46c9-b0fa-27886259ab37 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:02.338604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61264dbe-ae23-46c9-b0fa-27886259ab37 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:02.339010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61264dbe-ae23-46c9-b0fa-27886259ab37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:02.340731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61264dbe-ae23-46c9-b0fa-27886259ab37 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:02.346363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 482/955] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:47:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:02.354385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61264dbe-ae23-46c9-b0fa-27886259ab37 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:02.372597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acf9e8-0500-4d13-bae8-51fd6cf0053e None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:02.413769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad1ae2-c50c-482e-aab4-fbe03cad2766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:02.415552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad1ae2-c50c-482e-aab4-fbe03cad2766 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:02.416534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad1ae2-c50c-482e-aab4-fbe03cad2766 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:02.417026 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad1ae2-c50c-482e-aab4-fbe03cad2766 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:02.431944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad1ae2-c50c-482e-aab4-fbe03cad2766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-4dad1ae2-c50c-482e-aab4-fbe03cad2766 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:47:02.444199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:02.445159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:47:02.446088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:02.449872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 474/956] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 12:47:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:02.455119 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad1ae2-c50c-482e-aab4-fbe03cad2766 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:02.483865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61264dbe-ae23-46c9-b0fa-27886259ab37 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:02.486344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:02.536971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4192652-b726-43c1-ac57-263a4b718704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:02.537638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4192652-b726-43c1-ac57-263a4b718704 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:02.537921 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4192652-b726-43c1-ac57-263a4b718704 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:02.538195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4192652-b726-43c1-ac57-263a4b718704 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:02.538927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4192652-b726-43c1-ac57-263a4b718704 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:02.539232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4192652-b726-43c1-ac57-263a4b718704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:02.540604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4192652-b726-43c1-ac57-263a4b718704 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:02.550374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 483/957] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:47:02 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:02.580232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad1ae2-c50c-482e-aab4-fbe03cad2766 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:02.618218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e0e2c-221b-4023-bb4c-6c2591a39cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:02.619394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e0e2c-221b-4023-bb4c-6c2591a39cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:02.619893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e0e2c-221b-4023-bb4c-6c2591a39cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:02.620376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e0e2c-221b-4023-bb4c-6c2591a39cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:02.621358 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e0e2c-221b-4023-bb4c-6c2591a39cf6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:02.621879 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e0e2c-221b-4023-bb4c-6c2591a39cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:02.624178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e0e2c-221b-4023-bb4c-6c2591a39cf6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:02.634461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 475/958] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 12:47:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:02.640656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4192652-b726-43c1-ac57-263a4b718704 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:02.672631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:02.751206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4192652-b726-43c1-ac57-263a4b718704 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:02.797961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dfb2b-010d-4a92-ab8c-fa26f99650db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:02.798795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dfb2b-010d-4a92-ab8c-fa26f99650db None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:02.799074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dfb2b-010d-4a92-ab8c-fa26f99650db None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:02.799342 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dfb2b-010d-4a92-ab8c-fa26f99650db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:02.811562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dfb2b-010d-4a92-ab8c-fa26f99650db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:02.813053 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dfb2b-010d-4a92-ab8c-fa26f99650db None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-b74dfb2b-010d-4a92-ab8c-fa26f99650db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:47:02.830896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:02.831858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:02.832784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:47:02.832784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:47:02.832784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:02.832784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:02.832784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 484/959] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:02.837898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e0e2c-221b-4023-bb4c-6c2591a39cf6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:02.927917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:02.944206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e0e2c-221b-4023-bb4c-6c2591a39cf6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:02.979183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08812bfa-557d-41d6-8461-7b1e1ae0b264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:02.979856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08812bfa-557d-41d6-8461-7b1e1ae0b264 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:02.980137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08812bfa-557d-41d6-8461-7b1e1ae0b264 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:02.980411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08812bfa-557d-41d6-8461-7b1e1ae0b264 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:02.981131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08812bfa-557d-41d6-8461-7b1e1ae0b264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:02.981433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08812bfa-557d-41d6-8461-7b1e1ae0b264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:02.983454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08812bfa-557d-41d6-8461-7b1e1ae0b264 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:02.996933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 476/960] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:03.005992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dfb2b-010d-4a92-ab8c-fa26f99650db None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:03.087560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:03.111983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dfb2b-010d-4a92-ab8c-fa26f99650db None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:03.156925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d219d5-c30e-46e8-ba6c-6365f5bf0ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:03.157740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d219d5-c30e-46e8-ba6c-6365f5bf0ada None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:03.158020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d219d5-c30e-46e8-ba6c-6365f5bf0ada None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:03.158311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d219d5-c30e-46e8-ba6c-6365f5bf0ada None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:03.166647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d219d5-c30e-46e8-ba6c-6365f5bf0ada None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:03.168098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d219d5-c30e-46e8-ba6c-6365f5bf0ada None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-33d219d5-c30e-46e8-ba6c-6365f5bf0ada None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:47:03.183995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:03.185002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:03.186180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:03.186180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:03.186180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:47:03.186180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:47:03.186180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:47:03.186180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:47:03.186180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:47:03.186180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:47:03.186180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 12:47:03.186180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:03.186180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 485/961] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 12:47:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:03.191771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08812bfa-557d-41d6-8461-7b1e1ae0b264 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:03.300723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08812bfa-557d-41d6-8461-7b1e1ae0b264 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:03.313648 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f0fc-1575-453e-9bd5-e62174183964 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:03.314295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f0fc-1575-453e-9bd5-e62174183964 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:03.314777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f0fc-1575-453e-9bd5-e62174183964 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:03.316566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 486/962] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:03.321804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52a2e6-0d36-4707-a072-281ea9d3ed0e None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:03.322338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52a2e6-0d36-4707-a072-281ea9d3ed0e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:03.322830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52a2e6-0d36-4707-a072-281ea9d3ed0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:03.337838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6829e-2c0e-45bc-bfca-658aaa8109fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:03.338560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6829e-2c0e-45bc-bfca-658aaa8109fb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:03.338850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6829e-2c0e-45bc-bfca-658aaa8109fb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:03.339129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6829e-2c0e-45bc-bfca-658aaa8109fb None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:03.339975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6829e-2c0e-45bc-bfca-658aaa8109fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:03.340289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6829e-2c0e-45bc-bfca-658aaa8109fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:03.341717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6829e-2c0e-45bc-bfca-658aaa8109fb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:03.342967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ad52a2e6-0d36-4707-a072-281ea9d3ed0e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:03.344553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ad52a2e6-0d36-4707-a072-281ea9d3ed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:03.373378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 477/963] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 12:47:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:03.374920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6829e-2c0e-45bc-bfca-658aaa8109fb None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:03.379762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:03.447771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 487/964] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:47:03.470605 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52a2e6-0d36-4707-a072-281ea9d3ed0e None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:03.504433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6829e-2c0e-45bc-bfca-658aaa8109fb None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:03.551599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e7a1d-4627-4c24-aac5-95db7093a984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:03.552677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e7a1d-4627-4c24-aac5-95db7093a984 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:03.553110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e7a1d-4627-4c24-aac5-95db7093a984 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:03.553639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e7a1d-4627-4c24-aac5-95db7093a984 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:03.556805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:03.599978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52a2e6-0d36-4707-a072-281ea9d3ed0e None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:03.643246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e7a1d-4627-4c24-aac5-95db7093a984 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:03.645901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e7a1d-4627-4c24-aac5-95db7093a984 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:03.651594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:03.653481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d2182-9ca3-45dd-880d-7f35aea97e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:03.654427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d2182-9ca3-45dd-880d-7f35aea97e34 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:03.654716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d2182-9ca3-45dd-880d-7f35aea97e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:03.655140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d2182-9ca3-45dd-880d-7f35aea97e34 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:03.655807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d2182-9ca3-45dd-880d-7f35aea97e34 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:03.657193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d2182-9ca3-45dd-880d-7f35aea97e34 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-c42d2182-9ca3-45dd-880d-7f35aea97e34 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:03.674801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:03.675829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:03.676696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:03.681138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 488/965] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 12:47:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:47:03.682418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d2182-9ca3-45dd-880d-7f35aea97e34 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:03.767231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 478/966] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:47:03 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:47:03.767231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f13256-2e2f-436c-9a9f-81b202d51bc3 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:03.767231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f13256-2e2f-436c-9a9f-81b202d51bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:03.767231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f13256-2e2f-436c-9a9f-81b202d51bc3 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:03.767231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 479/967] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:03.770570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f13256-2e2f-436c-9a9f-81b202d51bc3 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:03.805713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d2182-9ca3-45dd-880d-7f35aea97e34 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:03.837568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:03.843051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd095b-6068-49c0-90cf-ad05d44afc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:03.843725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd095b-6068-49c0-90cf-ad05d44afc8d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:03.844009 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd095b-6068-49c0-90cf-ad05d44afc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:03.844284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd095b-6068-49c0-90cf-ad05d44afc8d None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:03.845058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd095b-6068-49c0-90cf-ad05d44afc8d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:03.845365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd095b-6068-49c0-90cf-ad05d44afc8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:03.846874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd095b-6068-49c0-90cf-ad05d44afc8d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:03.856563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 489/968] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 12:47:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:03.857716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fc144-f319-48ae-b52c-464acfa66265 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:03.858006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fc144-f319-48ae-b52c-464acfa66265 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:03.858312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fc144-f319-48ae-b52c-464acfa66265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:03.874583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-1c4fc144-f319-48ae-b52c-464acfa66265 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:03.875770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1c4fc144-f319-48ae-b52c-464acfa66265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:03.884613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f13256-2e2f-436c-9a9f-81b202d51bc3 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:03.887698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:03.924074 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b41e6-2ca8-4aaf-a59e-1be86364bfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:03.924969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b41e6-2ca8-4aaf-a59e-1be86364bfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:03.925173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b41e6-2ca8-4aaf-a59e-1be86364bfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:03.925456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b41e6-2ca8-4aaf-a59e-1be86364bfd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:03.936312 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b41e6-2ca8-4aaf-a59e-1be86364bfd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:03.937816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b41e6-2ca8-4aaf-a59e-1be86364bfd6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:03.946467 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 490/969] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:03.947279 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fc144-f319-48ae-b52c-464acfa66265 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-562b41e6-2ca8-4aaf-a59e-1be86364bfd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:47:03.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:03.959274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:03.960197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:47:03.960197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:47:03.960197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:03.960197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:03.960197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 480/970] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:03.962367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b41e6-2ca8-4aaf-a59e-1be86364bfd6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:04.057624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fc144-f319-48ae-b52c-464acfa66265 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:04.074671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b41e6-2ca8-4aaf-a59e-1be86364bfd6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:04.084292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:04.094947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422bc64-cc9a-46db-8aba-7d3032ec5d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:04.095619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422bc64-cc9a-46db-8aba-7d3032ec5d2f None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:04.095911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422bc64-cc9a-46db-8aba-7d3032ec5d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:04.096187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422bc64-cc9a-46db-8aba-7d3032ec5d2f None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:04.096903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422bc64-cc9a-46db-8aba-7d3032ec5d2f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:04.097211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422bc64-cc9a-46db-8aba-7d3032ec5d2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:04.098643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422bc64-cc9a-46db-8aba-7d3032ec5d2f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:04.103618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 491/971] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 12:47:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:04.104357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422bc64-cc9a-46db-8aba-7d3032ec5d2f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:04.115460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905ba8-3684-4796-ad59-3c470e0a5457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:04.116395 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905ba8-3684-4796-ad59-3c470e0a5457 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:04.116738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905ba8-3684-4796-ad59-3c470e0a5457 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:04.117080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905ba8-3684-4796-ad59-3c470e0a5457 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:04.117874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905ba8-3684-4796-ad59-3c470e0a5457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:04.118185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905ba8-3684-4796-ad59-3c470e0a5457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:04.119572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905ba8-3684-4796-ad59-3c470e0a5457 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:04.132724 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 481/972] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:04.133954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905ba8-3684-4796-ad59-3c470e0a5457 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:04.156699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:04.214170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422bc64-cc9a-46db-8aba-7d3032ec5d2f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:04.249745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42a5d2-9a5e-4851-bed6-7523e91e8799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:04.250453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42a5d2-9a5e-4851-bed6-7523e91e8799 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:04.250744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42a5d2-9a5e-4851-bed6-7523e91e8799 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:04.251023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42a5d2-9a5e-4851-bed6-7523e91e8799 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:04.251730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42a5d2-9a5e-4851-bed6-7523e91e8799 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:04.253295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42a5d2-9a5e-4851-bed6-7523e91e8799 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:04.254626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: WARNING keystone.server.flask.common [None req-9d42a5d2-9a5e-4851-bed6-7523e91e8799 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 24 12:47:04.257737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905ba8-3684-4796-ad59-3c470e0a5457 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:04.267326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.sql.core [None req-9d42a5d2-9a5e-4851-bed6-7523e91e8799 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 12:47:04.267326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 24 12:47:04.267326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [parameters: {'user_id': '446b99c2493043ec8c20814396d58cdc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 24 12:47:04.267326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80090) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-9d42a5d2-9a5e-4851-bed6-7523e91e8799 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application result = self._query(query) May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application conn.query(q) May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application result.read() May 24 12:47:04.275527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 12:47:04.276464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application next(self.gen) May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application yield resource May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application next(self.gen) May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self.session.rollback() May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self.force_reraise() May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise self.value May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application session.commit() May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 12:47:04.277473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._prepare_impl() May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self.session.flush() May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._flush(objects) May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application compat.raise_( May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application flush_context.execute() May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rec.execute(self) May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 12:47:04.279043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application result = connection._execute_20( May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 12:47:04.279944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application result = self._query(query) May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application conn.query(q) May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application result.read() May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 12:47:04.280811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [parameters: {'user_id': '446b99c2493043ec8c20814396d58cdc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:04.281965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 24 12:47:04.282914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 24 12:47:04.284064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 24 12:47:04.284064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 24 12:47:04.284064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 12:47:04.284064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 12:47:04.284064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 24 12:47:04.284064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:04.284064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 492/973] 10.222.0.45 () {60 vars in 1131 bytes} [Tue May 24 12:47:04 2022] POST /identity/v3/users => generated 160 bytes in 173 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:04.284064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42a5d2-9a5e-4851-bed6-7523e91e8799 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:04.303475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256fbc-e453-4219-8c6e-542139123463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:04.304333 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256fbc-e453-4219-8c6e-542139123463 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:04.304727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256fbc-e453-4219-8c6e-542139123463 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:04.305071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256fbc-e453-4219-8c6e-542139123463 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:04.305876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1256fbc-e453-4219-8c6e-542139123463 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:04.307551 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1256fbc-e453-4219-8c6e-542139123463 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:04.335291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 482/974] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 12:47:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:47:04.335291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256fbc-e453-4219-8c6e-542139123463 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:04.349274 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:04.416868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42a5d2-9a5e-4851-bed6-7523e91e8799 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:04.435940 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:04.477387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d3837-9856-481c-9753-ad51e8d0ce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:04.478665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d3837-9856-481c-9753-ad51e8d0ce76 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:04.479394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d3837-9856-481c-9753-ad51e8d0ce76 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:04.479858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d3837-9856-481c-9753-ad51e8d0ce76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:04.480954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256fbc-e453-4219-8c6e-542139123463 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:04.490499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d3837-9856-481c-9753-ad51e8d0ce76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:04.492949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d3837-9856-481c-9753-ad51e8d0ce76 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-b21d3837-9856-481c-9753-ad51e8d0ce76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:47:04.512636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:04.514180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:04.515139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:47:04.515139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:47:04.515139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:47:04.515139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:47:04.515139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:47:04.515139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:47:04.515139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:47:04.515139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:04.518442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 493/975] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:47:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:04.521615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d3837-9856-481c-9753-ad51e8d0ce76 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:04.531371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794e421-49d0-4437-a6ec-c959f082e889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:04.532572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794e421-49d0-4437-a6ec-c959f082e889 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:04.533899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794e421-49d0-4437-a6ec-c959f082e889 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:04.534192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794e421-49d0-4437-a6ec-c959f082e889 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:04.543647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794e421-49d0-4437-a6ec-c959f082e889 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:04.545687 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794e421-49d0-4437-a6ec-c959f082e889 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-b794e421-49d0-4437-a6ec-c959f082e889 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:47:04.560410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:04.561976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:04.563071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:04.563071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:04.563071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:47:04.563071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:47:04.563071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:47:04.563071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:47:04.563071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:47:04.563071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:47:04.563071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 12:47:04.563071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:04.563071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 483/976] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:47:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:04.567949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794e421-49d0-4437-a6ec-c959f082e889 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:04.652257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d3837-9856-481c-9753-ad51e8d0ce76 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:04.666894 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:04.691975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794e421-49d0-4437-a6ec-c959f082e889 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:04.706000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03239e-e66e-4932-8240-400ee8086256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:04.707051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03239e-e66e-4932-8240-400ee8086256 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:04.707334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03239e-e66e-4932-8240-400ee8086256 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:04.707608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03239e-e66e-4932-8240-400ee8086256 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:04.708395 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03239e-e66e-4932-8240-400ee8086256 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:04.708702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03239e-e66e-4932-8240-400ee8086256 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:04.710145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03239e-e66e-4932-8240-400ee8086256 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:04.728633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:04.729257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 494/977] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:47:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:04.735969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03239e-e66e-4932-8240-400ee8086256 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:04.742456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc74e5d-9e01-48bd-8aeb-2caf4640aba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:04.743582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc74e5d-9e01-48bd-8aeb-2caf4640aba3 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:04.746801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc74e5d-9e01-48bd-8aeb-2caf4640aba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:04.747083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc74e5d-9e01-48bd-8aeb-2caf4640aba3 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:04.748536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc74e5d-9e01-48bd-8aeb-2caf4640aba3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:04.748913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc74e5d-9e01-48bd-8aeb-2caf4640aba3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:04.751165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc74e5d-9e01-48bd-8aeb-2caf4640aba3 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:04.792468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 484/978] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 12:47:04 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:04.858836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03239e-e66e-4932-8240-400ee8086256 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:04.899169 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeaaab-dc63-40a6-ac0d-040c007af6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:04.900225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeaaab-dc63-40a6-ac0d-040c007af6aa None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:04.900721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeaaab-dc63-40a6-ac0d-040c007af6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:04.901244 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeaaab-dc63-40a6-ac0d-040c007af6aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:04.902414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeaaab-dc63-40a6-ac0d-040c007af6aa None admin] RBAC: Adding query filter params (role.id=b138ce6a3b514198ba4a9bdddb99ab81, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=122fdcf8f13348cf858555a565e5cf09) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:04.902946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeaaab-dc63-40a6-ac0d-040c007af6aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:04.905193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeaaab-dc63-40a6-ac0d-040c007af6aa None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:04.912873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 495/979] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:47:04 2022] GET /identity/v3/role_assignments?user.id=122fdcf8f13348cf858555a565e5cf09&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=b138ce6a3b514198ba4a9bdddb99ab81 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:04.935076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666f2a2-d733-4c82-aad5-cb8a389ae865 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:04.935401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666f2a2-d733-4c82-aad5-cb8a389ae865 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:04.936321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666f2a2-d733-4c82-aad5-cb8a389ae865 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:04.939538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 485/980] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:04.944478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a709827-e81c-499a-89db-9757744ef53c None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:04.944816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a709827-e81c-499a-89db-9757744ef53c None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:04.945119 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a709827-e81c-499a-89db-9757744ef53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:04.965224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7a709827-e81c-499a-89db-9757744ef53c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:04.966905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7a709827-e81c-499a-89db-9757744ef53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:04.998988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:05.058711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 496/981] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:05.082631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666f2a2-d733-4c82-aad5-cb8a389ae865 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:05.204063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666f2a2-d733-4c82-aad5-cb8a389ae865 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:05.245087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d493166-c71c-4c3a-8dfd-7b36d44eaef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:05.245772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d493166-c71c-4c3a-8dfd-7b36d44eaef7 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:05.246059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d493166-c71c-4c3a-8dfd-7b36d44eaef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:05.246368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d493166-c71c-4c3a-8dfd-7b36d44eaef7 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:05.247113 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d493166-c71c-4c3a-8dfd-7b36d44eaef7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:05.249070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d493166-c71c-4c3a-8dfd-7b36d44eaef7 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:05.264778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 486/982] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:47:05 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:47:06.689849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ccb95-750f-406c-9b66-200106079262 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:06.690532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ccb95-750f-406c-9b66-200106079262 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:06.690532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ccb95-750f-406c-9b66-200106079262 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:06.691794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 497/983] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:06.696967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bb145-fdd4-4ac9-89b8-360c52577f2a None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:06.697405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bb145-fdd4-4ac9-89b8-360c52577f2a None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:06.697560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bb145-fdd4-4ac9-89b8-360c52577f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:06.714605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9d3bb145-fdd4-4ac9-89b8-360c52577f2a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:06.715801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9d3bb145-fdd4-4ac9-89b8-360c52577f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:06.760283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:06.798330 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 487/984] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:06.815742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ccb95-750f-406c-9b66-200106079262 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:06.921506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ccb95-750f-406c-9b66-200106079262 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:06.955174 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bb8c2-4bb6-40ee-b0a8-01dfeebb043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:06.955904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bb8c2-4bb6-40ee-b0a8-01dfeebb043f None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:06.956181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bb8c2-4bb6-40ee-b0a8-01dfeebb043f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:06.956450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bb8c2-4bb6-40ee-b0a8-01dfeebb043f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:06.957095 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bb8c2-4bb6-40ee-b0a8-01dfeebb043f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:06.958557 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bb8c2-4bb6-40ee-b0a8-01dfeebb043f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-d65bb8c2-4bb6-40ee-b0a8-01dfeebb043f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:06.971956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:06.972964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:06.973839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:06.975392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 498/985] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 12:47:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:06.979184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bb145-fdd4-4ac9-89b8-360c52577f2a None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:07.084600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bb145-fdd4-4ac9-89b8-360c52577f2a None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:07.118840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cea683-42cc-47fb-8a47-27aa7927ee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:07.119515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cea683-42cc-47fb-8a47-27aa7927ee95 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.119797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cea683-42cc-47fb-8a47-27aa7927ee95 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.120073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cea683-42cc-47fb-8a47-27aa7927ee95 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.120773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cea683-42cc-47fb-8a47-27aa7927ee95 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:07.121077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cea683-42cc-47fb-8a47-27aa7927ee95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:07.122467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cea683-42cc-47fb-8a47-27aa7927ee95 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:07.131800 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 488/986] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 12:47:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:07.133059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bb8c2-4bb6-40ee-b0a8-01dfeebb043f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:07.161731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:07.212251 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ec352-c5a2-4299-81a0-ddb41b9970e2 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.212548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ec352-c5a2-4299-81a0-ddb41b9970e2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.212719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ec352-c5a2-4299-81a0-ddb41b9970e2 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.214097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 489/987] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:47:07.219523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc65c3-9ec1-4df5-83cc-9b366ce60fc5 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.219818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc65c3-9ec1-4df5-83cc-9b366ce60fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.220102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc65c3-9ec1-4df5-83cc-9b366ce60fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.236832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-67cc65c3-9ec1-4df5-83cc-9b366ce60fc5 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:07.238015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-67cc65c3-9ec1-4df5-83cc-9b366ce60fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:07.239795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bb8c2-4bb6-40ee-b0a8-01dfeebb043f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:07.266733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:07.276024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df3e09-858a-4488-8ba1-e1d4546f1109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:07.276952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df3e09-858a-4488-8ba1-e1d4546f1109 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.277382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df3e09-858a-4488-8ba1-e1d4546f1109 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.277799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df3e09-858a-4488-8ba1-e1d4546f1109 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.278702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df3e09-858a-4488-8ba1-e1d4546f1109 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:07.279765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df3e09-858a-4488-8ba1-e1d4546f1109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:07.281208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df3e09-858a-4488-8ba1-e1d4546f1109 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:07.287387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 499/988] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 12:47:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:07.293101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df3e09-858a-4488-8ba1-e1d4546f1109 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:07.319874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 490/989] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:47:07.342679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc65c3-9ec1-4df5-83cc-9b366ce60fc5 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:07.406762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:07.418092 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df3e09-858a-4488-8ba1-e1d4546f1109 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:07.451919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc65c3-9ec1-4df5-83cc-9b366ce60fc5 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:07.454340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c98804-0380-4ae8-b242-000d3e3a333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:07.455011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c98804-0380-4ae8-b242-000d3e3a333a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.455298 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c98804-0380-4ae8-b242-000d3e3a333a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.455576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c98804-0380-4ae8-b242-000d3e3a333a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.456391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c98804-0380-4ae8-b242-000d3e3a333a None admin] RBAC: Adding query filter params (interface=admin, service_id=6efb98fbee1c4569a51726c5c163dce2, region_id=RegionOne) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:07.456707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c98804-0380-4ae8-b242-000d3e3a333a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:07.458089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c98804-0380-4ae8-b242-000d3e3a333a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:07.467672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 500/990] 10.222.0.45 () {56 vars in 1249 bytes} [Tue May 24 12:47:07 2022] GET /identity/v3/endpoints?service_id=6efb98fbee1c4569a51726c5c163dce2&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:07.468919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c98804-0380-4ae8-b242-000d3e3a333a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:07.493610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0434386-7d50-454a-a92c-8d107b3300c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:07.494453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0434386-7d50-454a-a92c-8d107b3300c5 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.494742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0434386-7d50-454a-a92c-8d107b3300c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.495021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0434386-7d50-454a-a92c-8d107b3300c5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.495680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0434386-7d50-454a-a92c-8d107b3300c5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:07.497034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0434386-7d50-454a-a92c-8d107b3300c5 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-b0434386-7d50-454a-a92c-8d107b3300c5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:07.509284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:07.510605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:07.512098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 491/991] 10.222.0.45 () {56 vars in 1105 bytes} [Tue May 24 12:47:07 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:47:07.513031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65eb5e-ddee-4603-80c1-ec6ce4f9ae77 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.513031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65eb5e-ddee-4603-80c1-ec6ce4f9ae77 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.513031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65eb5e-ddee-4603-80c1-ec6ce4f9ae77 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.514061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 492/992] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:07.516442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65eb5e-ddee-4603-80c1-ec6ce4f9ae77 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:07.545888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:07.574892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c98804-0380-4ae8-b242-000d3e3a333a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:07.607161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd77722-09d0-42d1-92be-087274e478a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:07.608410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77722-09d0-42d1-92be-087274e478a1 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.608691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77722-09d0-42d1-92be-087274e478a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.608965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77722-09d0-42d1-92be-087274e478a1 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.609638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd77722-09d0-42d1-92be-087274e478a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:07.611040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd77722-09d0-42d1-92be-087274e478a1 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:07.616213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65eb5e-ddee-4603-80c1-ec6ce4f9ae77 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:07.617168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 501/993] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 12:47:07 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 12:47:07.618360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292a029-ceb3-4adf-a1bd-76438febcc54 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.618643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292a029-ceb3-4adf-a1bd-76438febcc54 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.618928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292a029-ceb3-4adf-a1bd-76438febcc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.645183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-5292a029-ceb3-4adf-a1bd-76438febcc54 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:07.646564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5292a029-ceb3-4adf-a1bd-76438febcc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:07.654757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ef8e0-4608-43e3-87ba-7442345ce83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:07.655459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ef8e0-4608-43e3-87ba-7442345ce83b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.655750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ef8e0-4608-43e3-87ba-7442345ce83b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.656027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ef8e0-4608-43e3-87ba-7442345ce83b None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.656756 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ef8e0-4608-43e3-87ba-7442345ce83b None admin] RBAC: Adding query filter params (name=image) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:07.657068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ef8e0-4608-43e3-87ba-7442345ce83b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:07.658527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ef8e0-4608-43e3-87ba-7442345ce83b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:07.663229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:07.664314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 493/994] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:07.670209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ef8e0-4608-43e3-87ba-7442345ce83b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:07.735744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 502/995] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:07.758551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292a029-ceb3-4adf-a1bd-76438febcc54 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:07.781187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:07.789022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ef8e0-4608-43e3-87ba-7442345ce83b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:07.830736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97479408-4929-4af9-8a6c-45e4659d4336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:07.831413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97479408-4929-4af9-8a6c-45e4659d4336 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.831695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97479408-4929-4af9-8a6c-45e4659d4336 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.831968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97479408-4929-4af9-8a6c-45e4659d4336 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.832674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97479408-4929-4af9-8a6c-45e4659d4336 None admin] RBAC: Adding query filter params (type=image) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:07.832980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97479408-4929-4af9-8a6c-45e4659d4336 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:07.834360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97479408-4929-4af9-8a6c-45e4659d4336 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:07.839248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 494/996] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:07 2022] GET /identity/v3/services?type=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:07.868639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292a029-ceb3-4adf-a1bd-76438febcc54 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:07.907262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b6e40-ae0c-4938-9037-ae30a9921705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:07.908094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b6e40-ae0c-4938-9037-ae30a9921705 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:07.908376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b6e40-ae0c-4938-9037-ae30a9921705 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:07.908649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b6e40-ae0c-4938-9037-ae30a9921705 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:07.909313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b6e40-ae0c-4938-9037-ae30a9921705 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:07.910762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b6e40-ae0c-4938-9037-ae30a9921705 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-ff1b6e40-ae0c-4938-9037-ae30a9921705 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:07.923895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:07.925002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:07.925874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 503/997] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:47:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:47:07.931543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97479408-4929-4af9-8a6c-45e4659d4336 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:08.039303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97479408-4929-4af9-8a6c-45e4659d4336 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:08.073297 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662e2be-8912-404a-aab4-19e40176a4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:08.074033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662e2be-8912-404a-aab4-19e40176a4dd None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:08.074352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662e2be-8912-404a-aab4-19e40176a4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:08.074634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662e2be-8912-404a-aab4-19e40176a4dd None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:08.075392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662e2be-8912-404a-aab4-19e40176a4dd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:08.075700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662e2be-8912-404a-aab4-19e40176a4dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:08.077637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662e2be-8912-404a-aab4-19e40176a4dd None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:08.082797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 495/998] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:47:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:08.088571 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b6e40-ae0c-4938-9037-ae30a9921705 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:08.104975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:08.195227 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b6e40-ae0c-4938-9037-ae30a9921705 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:08.228708 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ecc4f-b553-4950-b50e-2c49eb290a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:08.229384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ecc4f-b553-4950-b50e-2c49eb290a97 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:08.229672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ecc4f-b553-4950-b50e-2c49eb290a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:08.229953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ecc4f-b553-4950-b50e-2c49eb290a97 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:08.230706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ecc4f-b553-4950-b50e-2c49eb290a97 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:08.231018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ecc4f-b553-4950-b50e-2c49eb290a97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:08.233016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ecc4f-b553-4950-b50e-2c49eb290a97 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:08.238216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 504/999] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:47:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:08.243724 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662e2be-8912-404a-aab4-19e40176a4dd None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:08.270733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:08.352450 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662e2be-8912-404a-aab4-19e40176a4dd None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:08.390121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ad56e-d767-42a9-84e2-b75238919d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:08.390831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ad56e-d767-42a9-84e2-b75238919d6e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:08.391122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ad56e-d767-42a9-84e2-b75238919d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:08.391402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ad56e-d767-42a9-84e2-b75238919d6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:08.392199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ad56e-d767-42a9-84e2-b75238919d6e None admin] RBAC: Adding query filter params (interface=public, service_id=e7a475e0fd75461486eea1feef9cedcf, region_id=RegionOne) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:08.392569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ad56e-d767-42a9-84e2-b75238919d6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:08.393944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ad56e-d767-42a9-84e2-b75238919d6e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:08.399575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 496/1000] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 12:47:08 2022] GET /identity/v3/endpoints?service_id=e7a475e0fd75461486eea1feef9cedcf&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:08.405030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ecc4f-b553-4950-b50e-2c49eb290a97 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:08.485607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:08.510504 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ecc4f-b553-4950-b50e-2c49eb290a97 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:08.544094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a518ec-130a-442c-99a3-e7762cd78e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:08.545333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a518ec-130a-442c-99a3-e7762cd78e8a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:08.545596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a518ec-130a-442c-99a3-e7762cd78e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:08.545876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a518ec-130a-442c-99a3-e7762cd78e8a None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:08.546585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a518ec-130a-442c-99a3-e7762cd78e8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:08.547968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a518ec-130a-442c-99a3-e7762cd78e8a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:08.558757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 505/1001] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 12:47:08 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 12:47:09.825182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e154e84-b549-4efc-a600-165c21dda4a2 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:09.825702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e154e84-b549-4efc-a600-165c21dda4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:09.825816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e154e84-b549-4efc-a600-165c21dda4a2 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:09.827381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 497/1002] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:09.832501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be818-2c7f-4227-8ec6-9f7250f40877 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:09.832934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be818-2c7f-4227-8ec6-9f7250f40877 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:09.833070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be818-2c7f-4227-8ec6-9f7250f40877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:09.851406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-607be818-2c7f-4227-8ec6-9f7250f40877 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:09.852657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-607be818-2c7f-4227-8ec6-9f7250f40877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:09.930316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 506/1003] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:09.947045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e154e84-b549-4efc-a600-165c21dda4a2 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:10.033283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:10.035648 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bad4fa-bee5-4462-ad68-a90f306b5a64 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:10.036259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bad4fa-bee5-4462-ad68-a90f306b5a64 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:10.036259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bad4fa-bee5-4462-ad68-a90f306b5a64 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:10.037801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 507/1004] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:10.042591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38bbb7-de57-402c-94d5-a2b52de58935 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:10.042881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38bbb7-de57-402c-94d5-a2b52de58935 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:10.043158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38bbb7-de57-402c-94d5-a2b52de58935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:10.051934 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e154e84-b549-4efc-a600-165c21dda4a2 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:10.064225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-3a38bbb7-de57-402c-94d5-a2b52de58935 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:10.069047 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3a38bbb7-de57-402c-94d5-a2b52de58935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:10.090162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f51ecd-cb8c-49f4-aa8d-a69d764b5179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:10.090933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51ecd-cb8c-49f4-aa8d-a69d764b5179 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:10.091221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51ecd-cb8c-49f4-aa8d-a69d764b5179 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:10.091507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51ecd-cb8c-49f4-aa8d-a69d764b5179 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:10.092170 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f51ecd-cb8c-49f4-aa8d-a69d764b5179 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:10.093552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f51ecd-cb8c-49f4-aa8d-a69d764b5179 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-71f51ecd-cb8c-49f4-aa8d-a69d764b5179 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:10.105863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:10.106893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:10.107786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 498/1005] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 12:47:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:10.113398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f51ecd-cb8c-49f4-aa8d-a69d764b5179 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:10.143647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:10.150108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 508/1006] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:47:10.172815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38bbb7-de57-402c-94d5-a2b52de58935 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:10.216770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f51ecd-cb8c-49f4-aa8d-a69d764b5179 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:10.250929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827d3ee-d7a0-4c62-8462-1f9157a74618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:10.251969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827d3ee-d7a0-4c62-8462-1f9157a74618 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:10.252249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827d3ee-d7a0-4c62-8462-1f9157a74618 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:10.252524 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827d3ee-d7a0-4c62-8462-1f9157a74618 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:10.253226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827d3ee-d7a0-4c62-8462-1f9157a74618 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:10.253533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827d3ee-d7a0-4c62-8462-1f9157a74618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:10.254912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827d3ee-d7a0-4c62-8462-1f9157a74618 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:10.260034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 499/1007] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 12:47:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:10.265650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827d3ee-d7a0-4c62-8462-1f9157a74618 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:10.282231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:10.283866 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38bbb7-de57-402c-94d5-a2b52de58935 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:10.326748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5b437-bfe3-49db-b98d-7bebfbb015fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:10.327676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5b437-bfe3-49db-b98d-7bebfbb015fb None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:10.327960 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5b437-bfe3-49db-b98d-7bebfbb015fb None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:10.328644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5b437-bfe3-49db-b98d-7bebfbb015fb None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:10.329640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5b437-bfe3-49db-b98d-7bebfbb015fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:10.331662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5b437-bfe3-49db-b98d-7bebfbb015fb None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:10.347318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 509/1008] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:47:10 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:47:10.379865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827d3ee-d7a0-4c62-8462-1f9157a74618 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:10.440779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1c2bb-b8de-4f58-975e-b6cfcc621bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:10.441488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1c2bb-b8de-4f58-975e-b6cfcc621bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:10.441776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1c2bb-b8de-4f58-975e-b6cfcc621bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:10.442057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1c2bb-b8de-4f58-975e-b6cfcc621bb7 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:10.442815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1c2bb-b8de-4f58-975e-b6cfcc621bb7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:10.443128 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1c2bb-b8de-4f58-975e-b6cfcc621bb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:10.444518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1c2bb-b8de-4f58-975e-b6cfcc621bb7 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:10.455931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 500/1009] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 12:47:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:10.457676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5b437-bfe3-49db-b98d-7bebfbb015fb None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:10.522180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:10.571174 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5b437-bfe3-49db-b98d-7bebfbb015fb None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:10.606404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb8f6a-25f9-48a0-b271-b7a4270c0d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:10.607820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb8f6a-25f9-48a0-b271-b7a4270c0d11 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:10.608246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb8f6a-25f9-48a0-b271-b7a4270c0d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:10.608701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb8f6a-25f9-48a0-b271-b7a4270c0d11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:10.609589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb8f6a-25f9-48a0-b271-b7a4270c0d11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:10.611141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb8f6a-25f9-48a0-b271-b7a4270c0d11 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:10.627984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 510/1010] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 12:47:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:47:10.831401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab51ff-82f2-42a3-90a9-c8364d18697d None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:10.831918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab51ff-82f2-42a3-90a9-c8364d18697d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:10.832024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab51ff-82f2-42a3-90a9-c8364d18697d None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:10.833347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 501/1011] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:10.839078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac28219-5aa4-46dd-b55b-721987eb85b9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:10.839520 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac28219-5aa4-46dd-b55b-721987eb85b9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:10.839632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac28219-5aa4-46dd-b55b-721987eb85b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:10.856659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-3ac28219-5aa4-46dd-b55b-721987eb85b9 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:10.857773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3ac28219-5aa4-46dd-b55b-721987eb85b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:10.934347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 511/1012] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:10 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:10.954254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab51ff-82f2-42a3-90a9-c8364d18697d None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:11.031343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:11.058052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab51ff-82f2-42a3-90a9-c8364d18697d None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:11.095922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef7789-77bb-479b-98f0-409ea87bb3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:11.096729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef7789-77bb-479b-98f0-409ea87bb3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:11.097017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef7789-77bb-479b-98f0-409ea87bb3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:11.097288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef7789-77bb-479b-98f0-409ea87bb3b1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:11.097943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef7789-77bb-479b-98f0-409ea87bb3b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:11.099339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef7789-77bb-479b-98f0-409ea87bb3b1 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-70ef7789-77bb-479b-98f0-409ea87bb3b1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:11.112014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:11.113039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:11.113905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 502/1013] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:47:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:11.120904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac28219-5aa4-46dd-b55b-721987eb85b9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:11.172682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:11.233043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac28219-5aa4-46dd-b55b-721987eb85b9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:11.266933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3f7f2-8b54-458b-97b8-84cde3cbd71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:11.267594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3f7f2-8b54-458b-97b8-84cde3cbd71f None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:11.267875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3f7f2-8b54-458b-97b8-84cde3cbd71f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:11.268146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3f7f2-8b54-458b-97b8-84cde3cbd71f None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:11.268844 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3f7f2-8b54-458b-97b8-84cde3cbd71f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:11.269146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3f7f2-8b54-458b-97b8-84cde3cbd71f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:11.270535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3f7f2-8b54-458b-97b8-84cde3cbd71f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:11.275479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 512/1014] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:47:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:11.281385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef7789-77bb-479b-98f0-409ea87bb3b1 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:11.384979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef7789-77bb-479b-98f0-409ea87bb3b1 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:11.421563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50982047-1c4c-47c2-8e11-573dfb84cac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:11.422232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50982047-1c4c-47c2-8e11-573dfb84cac0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:11.422544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50982047-1c4c-47c2-8e11-573dfb84cac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:11.422863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50982047-1c4c-47c2-8e11-573dfb84cac0 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:11.423630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50982047-1c4c-47c2-8e11-573dfb84cac0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:11.423934 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50982047-1c4c-47c2-8e11-573dfb84cac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:11.425289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50982047-1c4c-47c2-8e11-573dfb84cac0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:11.434781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 503/1015] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 12:47:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:11.436796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3f7f2-8b54-458b-97b8-84cde3cbd71f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:11.553202 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3f7f2-8b54-458b-97b8-84cde3cbd71f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:11.565216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:11.590507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4d3d2-72e9-480f-b81d-a20f653566e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:11.591399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4d3d2-72e9-480f-b81d-a20f653566e3 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:11.591876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4d3d2-72e9-480f-b81d-a20f653566e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:11.592294 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4d3d2-72e9-480f-b81d-a20f653566e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:11.593177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4d3d2-72e9-480f-b81d-a20f653566e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:11.594712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4d3d2-72e9-480f-b81d-a20f653566e3 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:11.614418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 513/1016] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 12:47:11 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:47:12.571664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68ef3-eaa8-4d51-9d40-44fcf9901fdc None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:12.572176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68ef3-eaa8-4d51-9d40-44fcf9901fdc None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:12.572254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68ef3-eaa8-4d51-9d40-44fcf9901fdc None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:12.573611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 504/1017] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:12.579326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34369147-035f-4176-9168-11e25c2c1bfc None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:12.579709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34369147-035f-4176-9168-11e25c2c1bfc None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:12.579885 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34369147-035f-4176-9168-11e25c2c1bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:12.597183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-34369147-035f-4176-9168-11e25c2c1bfc None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:12.598374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-34369147-035f-4176-9168-11e25c2c1bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:12.671022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 514/1018] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:12.693520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae68ef3-eaa8-4d51-9d40-44fcf9901fdc None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:12.696576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:12.828532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae68ef3-eaa8-4d51-9d40-44fcf9901fdc None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:12.862402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3f996-299c-4042-9616-ef000e9b994e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:12.863203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3f996-299c-4042-9616-ef000e9b994e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:12.863485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3f996-299c-4042-9616-ef000e9b994e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:12.863759 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3f996-299c-4042-9616-ef000e9b994e None admin] PATH_INFO: `/v3/services/image` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:12.864408 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3f996-299c-4042-9616-ef000e9b994e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:12.865757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3f996-299c-4042-9616-ef000e9b994e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-9fb3f996-299c-4042-9616-ef000e9b994e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:12.877498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:12.878571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 12:47:12.883481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:12.884417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514a4b-4c68-43b6-9ca3-f12baf395d40 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:12.884417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514a4b-4c68-43b6-9ca3-f12baf395d40 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:12.884417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514a4b-4c68-43b6-9ca3-f12baf395d40 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:12.884902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 505/1019] 10.222.0.45 () {56 vars in 1105 bytes} [Tue May 24 12:47:12 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:12.884998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 515/1020] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:12.885080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b9b1c-60d7-4b89-bbac-dd74c0bca3dd None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:12.885171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b9b1c-60d7-4b89-bbac-dd74c0bca3dd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:12.885385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b9b1c-60d7-4b89-bbac-dd74c0bca3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:12.887510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88514a4b-4c68-43b6-9ca3-f12baf395d40 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:12.905120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-bd5b9b1c-60d7-4b89-bbac-dd74c0bca3dd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:12.906534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-bd5b9b1c-60d7-4b89-bbac-dd74c0bca3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:12.968677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:12.978043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 506/1021] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:12.987065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:13.000182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88514a4b-4c68-43b6-9ca3-f12baf395d40 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:13.001009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b9b1c-60d7-4b89-bbac-dd74c0bca3dd None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:13.037621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140162-53ce-464b-af54-df3753aa8156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:13.038317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140162-53ce-464b-af54-df3753aa8156 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:13.038626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140162-53ce-464b-af54-df3753aa8156 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:13.038905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140162-53ce-464b-af54-df3753aa8156 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:13.039612 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3140162-53ce-464b-af54-df3753aa8156 None admin] RBAC: Adding query filter params (name=image) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:13.040041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3140162-53ce-464b-af54-df3753aa8156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:13.041415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3140162-53ce-464b-af54-df3753aa8156 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:13.046962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 516/1022] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:13.052123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140162-53ce-464b-af54-df3753aa8156 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:13.113979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b9b1c-60d7-4b89-bbac-dd74c0bca3dd None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:13.150452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356c4ee-7f0b-4c5a-b383-dee85a94b659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:13.151538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356c4ee-7f0b-4c5a-b383-dee85a94b659 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:13.151820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356c4ee-7f0b-4c5a-b383-dee85a94b659 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:13.152092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356c4ee-7f0b-4c5a-b383-dee85a94b659 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:13.153255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356c4ee-7f0b-4c5a-b383-dee85a94b659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:13.155627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356c4ee-7f0b-4c5a-b383-dee85a94b659 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:13.157292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 507/1023] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:13.163139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356c4ee-7f0b-4c5a-b383-dee85a94b659 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:13.169308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140162-53ce-464b-af54-df3753aa8156 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:13.205939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff940075-59a9-469d-b0d6-a95b2e7de1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:13.206636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff940075-59a9-469d-b0d6-a95b2e7de1ce None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:13.206918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff940075-59a9-469d-b0d6-a95b2e7de1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:13.207193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff940075-59a9-469d-b0d6-a95b2e7de1ce None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:13.207898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff940075-59a9-469d-b0d6-a95b2e7de1ce None admin] RBAC: Adding query filter params (type=image) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:13.208246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff940075-59a9-469d-b0d6-a95b2e7de1ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:13.209579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff940075-59a9-469d-b0d6-a95b2e7de1ce None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:13.215083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 517/1024] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:13.221074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff940075-59a9-469d-b0d6-a95b2e7de1ce None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:13.236729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:13.243755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:13.280008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356c4ee-7f0b-4c5a-b383-dee85a94b659 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:13.316416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb1cb0-119e-4642-a542-193118f7ed53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:13.317085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb1cb0-119e-4642-a542-193118f7ed53 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:13.317621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb1cb0-119e-4642-a542-193118f7ed53 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:13.317621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb1cb0-119e-4642-a542-193118f7ed53 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:13.318381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb1cb0-119e-4642-a542-193118f7ed53 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:13.319902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb1cb0-119e-4642-a542-193118f7ed53 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:13.327536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff940075-59a9-469d-b0d6-a95b2e7de1ce None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:13.351025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 508/1025] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:47:13 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:47:13.364826 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb1a27-471b-44c9-a7a7-dc401fca8958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:13.365520 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb1a27-471b-44c9-a7a7-dc401fca8958 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:13.365809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb1a27-471b-44c9-a7a7-dc401fca8958 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:13.366093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb1a27-471b-44c9-a7a7-dc401fca8958 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:13.366946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb1a27-471b-44c9-a7a7-dc401fca8958 None admin] RBAC: Adding query filter params (interface=public, service_id=6cd00fdfb7d24a81bb91fae99a879465, region_id=RegionOne) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:13.367267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb1a27-471b-44c9-a7a7-dc401fca8958 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:13.368709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb1a27-471b-44c9-a7a7-dc401fca8958 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:13.378326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 518/1026] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 12:47:13 2022] GET /identity/v3/endpoints?service_id=6cd00fdfb7d24a81bb91fae99a879465&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:13.381452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb1cb0-119e-4642-a542-193118f7ed53 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:13.497517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb1cb0-119e-4642-a542-193118f7ed53 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:13.516034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:13.538611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc4f2e-0aac-4819-a17c-5a19d6719816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:13.539298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc4f2e-0aac-4819-a17c-5a19d6719816 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:13.539584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc4f2e-0aac-4819-a17c-5a19d6719816 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:13.539863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc4f2e-0aac-4819-a17c-5a19d6719816 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:13.540559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc4f2e-0aac-4819-a17c-5a19d6719816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:13.541948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc4f2e-0aac-4819-a17c-5a19d6719816 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:13.554694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 509/1027] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 12:47:13 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:47:13.934397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe593b-e70c-450c-9dea-3b1146689bf6 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:13.934930 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe593b-e70c-450c-9dea-3b1146689bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:13.934930 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe593b-e70c-450c-9dea-3b1146689bf6 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:13.936991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 519/1028] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:13.941932 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bd2f9-9050-40f4-90a5-6e81b75094f8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:13.942366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bd2f9-9050-40f4-90a5-6e81b75094f8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:13.942533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bd2f9-9050-40f4-90a5-6e81b75094f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:13.959514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-713bd2f9-9050-40f4-90a5-6e81b75094f8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:13.960679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-713bd2f9-9050-40f4-90a5-6e81b75094f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:14.026380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 510/1029] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:14.049634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe593b-e70c-450c-9dea-3b1146689bf6 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:14.137088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:14.155926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe593b-e70c-450c-9dea-3b1146689bf6 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:14.191156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee9fe0-ff18-4313-ba09-b1575e3bc44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:14.191894 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee9fe0-ff18-4313-ba09-b1575e3bc44a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:14.192181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee9fe0-ff18-4313-ba09-b1575e3bc44a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:14.192458 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee9fe0-ff18-4313-ba09-b1575e3bc44a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:14.203292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee9fe0-ff18-4313-ba09-b1575e3bc44a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-05ee9fe0-ff18-4313-ba09-b1575e3bc44a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:47:14.211403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:14.212343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:47:14.213223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:14.214815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 520/1030] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 12:47:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:14.219069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bd2f9-9050-40f4-90a5-6e81b75094f8 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:14.318710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bd2f9-9050-40f4-90a5-6e81b75094f8 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:14.329824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:14.354554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebafc9-fa92-42fd-8d2c-f20eb1719c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:14.355227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebafc9-fa92-42fd-8d2c-f20eb1719c4d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:14.355515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebafc9-fa92-42fd-8d2c-f20eb1719c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:14.355793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebafc9-fa92-42fd-8d2c-f20eb1719c4d None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:14.356541 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebafc9-fa92-42fd-8d2c-f20eb1719c4d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:14.356848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebafc9-fa92-42fd-8d2c-f20eb1719c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:14.358233 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebafc9-fa92-42fd-8d2c-f20eb1719c4d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:14.370816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 511/1031] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 12:47:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:14.372193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee9fe0-ff18-4313-ba09-b1575e3bc44a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:14.484743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee9fe0-ff18-4313-ba09-b1575e3bc44a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:14.517620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a330f1-20f4-4336-8665-d0fc8ec3a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:14.518440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a330f1-20f4-4336-8665-d0fc8ec3a62d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:14.518728 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a330f1-20f4-4336-8665-d0fc8ec3a62d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:14.519003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a330f1-20f4-4336-8665-d0fc8ec3a62d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:14.529781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a330f1-20f4-4336-8665-d0fc8ec3a62d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:14.531283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a330f1-20f4-4336-8665-d0fc8ec3a62d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:14.533739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-e1a330f1-20f4-4336-8665-d0fc8ec3a62d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:47:14.550213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:14.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:14.552126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:47:14.552126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:47:14.552126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:14.552126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:14.552990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 521/1032] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:14.557408 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebafc9-fa92-42fd-8d2c-f20eb1719c4d None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:14.659994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebafc9-fa92-42fd-8d2c-f20eb1719c4d None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:14.695743 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196e827-0a3a-48e1-80b3-90d7505c9d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:14.696413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196e827-0a3a-48e1-80b3-90d7505c9d23 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:14.696695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196e827-0a3a-48e1-80b3-90d7505c9d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:14.696968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196e827-0a3a-48e1-80b3-90d7505c9d23 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:14.697678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196e827-0a3a-48e1-80b3-90d7505c9d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:14.697980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196e827-0a3a-48e1-80b3-90d7505c9d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:14.699364 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196e827-0a3a-48e1-80b3-90d7505c9d23 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:14.711135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 512/1033] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:14.716971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a330f1-20f4-4336-8665-d0fc8ec3a62d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:14.819020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a330f1-20f4-4336-8665-d0fc8ec3a62d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:14.852283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36b731-d666-4937-8750-5d707c868316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:14.853071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36b731-d666-4937-8750-5d707c868316 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:14.853354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36b731-d666-4937-8750-5d707c868316 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:14.853627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36b731-d666-4937-8750-5d707c868316 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:14.861582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36b731-d666-4937-8750-5d707c868316 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:14.863064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36b731-d666-4937-8750-5d707c868316 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-df36b731-d666-4937-8750-5d707c868316 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:47:14.876790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:14.877731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:14.883476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:14.883476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:14.883476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:47:14.883476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:47:14.883476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:47:14.883476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:47:14.883476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:47:14.883476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:47:14.883476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 12:47:14.883476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:14.883476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 522/1034] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 12:47:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:14.885481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196e827-0a3a-48e1-80b3-90d7505c9d23 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:14.917255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:14.985164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196e827-0a3a-48e1-80b3-90d7505c9d23 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:15.018718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed15160-60e7-462c-bd9d-043e428f6ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:15.019393 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed15160-60e7-462c-bd9d-043e428f6ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:15.019678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed15160-60e7-462c-bd9d-043e428f6ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:15.019956 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed15160-60e7-462c-bd9d-043e428f6ef9 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:15.020780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed15160-60e7-462c-bd9d-043e428f6ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:15.021092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed15160-60e7-462c-bd9d-043e428f6ef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:15.022542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed15160-60e7-462c-bd9d-043e428f6ef9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:15.039983 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 513/1035] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 12:47:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:15.045481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36b731-d666-4937-8750-5d707c868316 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:15.104933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:15.152951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36b731-d666-4937-8750-5d707c868316 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:15.188577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec642a1b-b7e5-4d65-bbdd-eb4317dd8bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:15.189238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec642a1b-b7e5-4d65-bbdd-eb4317dd8bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:15.189527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec642a1b-b7e5-4d65-bbdd-eb4317dd8bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:15.189805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec642a1b-b7e5-4d65-bbdd-eb4317dd8bc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:15.260649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec642a1b-b7e5-4d65-bbdd-eb4317dd8bc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:15.262118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec642a1b-b7e5-4d65-bbdd-eb4317dd8bc4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:15.316998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:15.367331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 523/1036] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:47:15 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 12:47:15.368785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed15160-60e7-462c-bd9d-043e428f6ef9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:15.466541 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:15.478795 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed15160-60e7-462c-bd9d-043e428f6ef9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:15.517243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1ab89-bb3d-479b-b682-ca8081ff6949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:15.518045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1ab89-bb3d-479b-b682-ca8081ff6949 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:15.518358 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1ab89-bb3d-479b-b682-ca8081ff6949 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:15.518640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1ab89-bb3d-479b-b682-ca8081ff6949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:15.528929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1ab89-bb3d-479b-b682-ca8081ff6949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:15.530385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1ab89-bb3d-479b-b682-ca8081ff6949 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-3ad1ab89-bb3d-479b-b682-ca8081ff6949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:47:15.548856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:15.549772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:15.551234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:47:15.551234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:47:15.551234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:15.551234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:15.554834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 514/1037] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:47:15.555791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec642a1b-b7e5-4d65-bbdd-eb4317dd8bc4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:15.662495 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec642a1b-b7e5-4d65-bbdd-eb4317dd8bc4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:15.664734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59f9a1-96b4-40e5-b1f0-1e517a17fa14 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:15.665026 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59f9a1-96b4-40e5-b1f0-1e517a17fa14 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:15.665307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59f9a1-96b4-40e5-b1f0-1e517a17fa14 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:15.666734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 515/1038] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:47:15.673627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592500a-a673-4d28-bd10-6aec0fb9b0bf None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:15.674002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592500a-a673-4d28-bd10-6aec0fb9b0bf None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:15.674700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592500a-a673-4d28-bd10-6aec0fb9b0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:15.691052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e592500a-a673-4d28-bd10-6aec0fb9b0bf None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:15.692374 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e592500a-a673-4d28-bd10-6aec0fb9b0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:15.703407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92833c8-1158-45e3-ba9e-acbe6047d21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:15.704407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92833c8-1158-45e3-ba9e-acbe6047d21a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:15.704841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92833c8-1158-45e3-ba9e-acbe6047d21a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:15.705129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92833c8-1158-45e3-ba9e-acbe6047d21a None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:15.705859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92833c8-1158-45e3-ba9e-acbe6047d21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:15.706176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92833c8-1158-45e3-ba9e-acbe6047d21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:15.707581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92833c8-1158-45e3-ba9e-acbe6047d21a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:15.724334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 524/1039] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:15.725384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92833c8-1158-45e3-ba9e-acbe6047d21a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:15.743624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:15.779262 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 516/1040] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:15.780591 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281437b6-607c-44f5-832e-1b0d43817463 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:15.781189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281437b6-607c-44f5-832e-1b0d43817463 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:15.781189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281437b6-607c-44f5-832e-1b0d43817463 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:15.782515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 517/1041] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:15.787260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455da8f-ebfd-488a-918c-626b8f8ef78f None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:15.787837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455da8f-ebfd-488a-918c-626b8f8ef78f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:15.787837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455da8f-ebfd-488a-918c-626b8f8ef78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:15.805145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1455da8f-ebfd-488a-918c-626b8f8ef78f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:15.806337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1455da8f-ebfd-488a-918c-626b8f8ef78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:15.837080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92833c8-1158-45e3-ba9e-acbe6047d21a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:15.870494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfaf11-100d-4056-8e9a-bc309848eb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:15.871284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfaf11-100d-4056-8e9a-bc309848eb5a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:15.871566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfaf11-100d-4056-8e9a-bc309848eb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:15.871848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfaf11-100d-4056-8e9a-bc309848eb5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:15.874355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 518/1042] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:47:15.875180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455da8f-ebfd-488a-918c-626b8f8ef78f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:15.877272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:15.879196 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfaf11-100d-4056-8e9a-bc309848eb5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:15.880772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfaf11-100d-4056-8e9a-bc309848eb5a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-34cfaf11-100d-4056-8e9a-bc309848eb5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:47:15.895743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:15.896803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:15.897836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:47:15.897836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:47:15.897836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:47:15.897836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:47:15.897836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:47:15.897836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:47:15.897836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:47:15.897836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:15.897836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 525/1043] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:15.897836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfaf11-100d-4056-8e9a-bc309848eb5a None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:15.972546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:15.976534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455da8f-ebfd-488a-918c-626b8f8ef78f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:15.996112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfaf11-100d-4056-8e9a-bc309848eb5a None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:16.010009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cf495-4ca1-46ea-90c3-e0908c7005e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:16.010835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cf495-4ca1-46ea-90c3-e0908c7005e7 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:16.011118 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cf495-4ca1-46ea-90c3-e0908c7005e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:16.011395 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cf495-4ca1-46ea-90c3-e0908c7005e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:16.012565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cf495-4ca1-46ea-90c3-e0908c7005e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:16.014158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cf495-4ca1-46ea-90c3-e0908c7005e7 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:16.015803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 519/1044] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:16.016551 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cf495-4ca1-46ea-90c3-e0908c7005e7 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:16.032889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433430e-9e67-4dbf-a958-9cf2dc280ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:16.033678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433430e-9e67-4dbf-a958-9cf2dc280ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:16.033963 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433430e-9e67-4dbf-a958-9cf2dc280ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:16.034240 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433430e-9e67-4dbf-a958-9cf2dc280ec5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:16.034923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433430e-9e67-4dbf-a958-9cf2dc280ec5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:16.036294 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433430e-9e67-4dbf-a958-9cf2dc280ec5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-9433430e-9e67-4dbf-a958-9cf2dc280ec5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:16.047690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:16.049139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:16.049993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 526/1045] 10.222.0.45 () {56 vars in 1105 bytes} [Tue May 24 12:47:15 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:16.050840 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433430e-9e67-4dbf-a958-9cf2dc280ec5 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:16.089718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:16.130053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cf495-4ca1-46ea-90c3-e0908c7005e7 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:16.150013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433430e-9e67-4dbf-a958-9cf2dc280ec5 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:16.164058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a895b32-0f4f-4d3e-a79a-6b4357eb24fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:16.164803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a895b32-0f4f-4d3e-a79a-6b4357eb24fd None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:16.165088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a895b32-0f4f-4d3e-a79a-6b4357eb24fd None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:16.165360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a895b32-0f4f-4d3e-a79a-6b4357eb24fd None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:16.166139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a895b32-0f4f-4d3e-a79a-6b4357eb24fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:16.166483 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a895b32-0f4f-4d3e-a79a-6b4357eb24fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:16.167918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a895b32-0f4f-4d3e-a79a-6b4357eb24fd None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:16.180325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 520/1046] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:16.181122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a895b32-0f4f-4d3e-a79a-6b4357eb24fd None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:16.183895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a434ea0-7425-41a7-ae8f-ef2763f1c5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:16.184569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a434ea0-7425-41a7-ae8f-ef2763f1c5b4 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:16.184853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a434ea0-7425-41a7-ae8f-ef2763f1c5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:16.185668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a434ea0-7425-41a7-ae8f-ef2763f1c5b4 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:16.186408 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a434ea0-7425-41a7-ae8f-ef2763f1c5b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:16.187841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a434ea0-7425-41a7-ae8f-ef2763f1c5b4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:16.197481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:16.210561 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 527/1047] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:47:16 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:47:16.211336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a434ea0-7425-41a7-ae8f-ef2763f1c5b4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:16.314526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a895b32-0f4f-4d3e-a79a-6b4357eb24fd None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:16.325194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a434ea0-7425-41a7-ae8f-ef2763f1c5b4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:16.346900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:16.353924 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81993137-c090-415e-be2d-aafbe37da97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:16.354638 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993137-c090-415e-be2d-aafbe37da97d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:16.354925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993137-c090-415e-be2d-aafbe37da97d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:16.355206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993137-c090-415e-be2d-aafbe37da97d None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:16.355927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993137-c090-415e-be2d-aafbe37da97d None admin] RBAC: Adding query filter params (name=image) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:16.356238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993137-c090-415e-be2d-aafbe37da97d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:16.358148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993137-c090-415e-be2d-aafbe37da97d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:16.368229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 521/1048] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:16.368425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd9a51-19eb-4aad-8d73-543bac0275eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:16.368425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd9a51-19eb-4aad-8d73-543bac0275eb None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:16.368425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd9a51-19eb-4aad-8d73-543bac0275eb None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:16.368425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd9a51-19eb-4aad-8d73-543bac0275eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:16.368730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd9a51-19eb-4aad-8d73-543bac0275eb None admin] RBAC: Adding query filter params (role.id=b138ce6a3b514198ba4a9bdddb99ab81, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=05b2edb6fcb7496bb55650080a9da172) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:16.368730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd9a51-19eb-4aad-8d73-543bac0275eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:16.369359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81993137-c090-415e-be2d-aafbe37da97d None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:16.370062 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd9a51-19eb-4aad-8d73-543bac0275eb None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:16.386313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 528/1049] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:47:16 2022] GET /identity/v3/role_assignments?user.id=05b2edb6fcb7496bb55650080a9da172&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=b138ce6a3b514198ba4a9bdddb99ab81 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:16.497514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81993137-c090-415e-be2d-aafbe37da97d None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:16.534334 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66485bdb-8c1d-4ef5-8f18-52db466f50c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:16.535030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66485bdb-8c1d-4ef5-8f18-52db466f50c0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:16.535315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66485bdb-8c1d-4ef5-8f18-52db466f50c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:16.535593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66485bdb-8c1d-4ef5-8f18-52db466f50c0 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:16.536313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66485bdb-8c1d-4ef5-8f18-52db466f50c0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:16.536623 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66485bdb-8c1d-4ef5-8f18-52db466f50c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:16.538006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66485bdb-8c1d-4ef5-8f18-52db466f50c0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:16.549013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 522/1050] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:16.550701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd9a51-19eb-4aad-8d73-543bac0275eb None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:16.614098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:16.658384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd9a51-19eb-4aad-8d73-543bac0275eb None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:16.692964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21157f1c-e8dc-41b7-8065-0f6d497f85ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:16.693682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21157f1c-e8dc-41b7-8065-0f6d497f85ea None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:16.693972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21157f1c-e8dc-41b7-8065-0f6d497f85ea None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:16.694245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21157f1c-e8dc-41b7-8065-0f6d497f85ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:16.695031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21157f1c-e8dc-41b7-8065-0f6d497f85ea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:16.696411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21157f1c-e8dc-41b7-8065-0f6d497f85ea None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:16.722327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 529/1051] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 12:47:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:47:18.518102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ec18-5290-47e4-b5b0-ae260ca950f2 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:18.518676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ec18-5290-47e4-b5b0-ae260ca950f2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:18.518759 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ec18-5290-47e4-b5b0-ae260ca950f2 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:18.520121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 523/1052] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:18.525379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f386a0-675f-4fe2-87cf-96d51161530e None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:18.525779 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f386a0-675f-4fe2-87cf-96d51161530e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:18.525961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f386a0-675f-4fe2-87cf-96d51161530e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:18.543419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e4f386a0-675f-4fe2-87cf-96d51161530e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:18.545205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e4f386a0-675f-4fe2-87cf-96d51161530e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:18.559693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387fa13-a2b6-4bd0-9eb5-f95c4d52fc42 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:18.560030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387fa13-a2b6-4bd0-9eb5-f95c4d52fc42 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:18.560307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387fa13-a2b6-4bd0-9eb5-f95c4d52fc42 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:18.561796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 524/1053] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:18.566415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171223c-6fd9-4f01-b258-c67eb6187801 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:18.566704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171223c-6fd9-4f01-b258-c67eb6187801 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:18.566980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171223c-6fd9-4f01-b258-c67eb6187801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:18.583824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e171223c-6fd9-4f01-b258-c67eb6187801 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:18.584996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e171223c-6fd9-4f01-b258-c67eb6187801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:18.623295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 530/1054] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:18.637244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:18.646808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f386a0-675f-4fe2-87cf-96d51161530e None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:18.652743 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 525/1055] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:47:18.674825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171223c-6fd9-4f01-b258-c67eb6187801 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:18.689759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:18.752810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f386a0-675f-4fe2-87cf-96d51161530e None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:18.785749 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171223c-6fd9-4f01-b258-c67eb6187801 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:18.791182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c109e-f356-4922-be02-d724eecc1442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:18.791988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c109e-f356-4922-be02-d724eecc1442 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:18.792275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c109e-f356-4922-be02-d724eecc1442 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:18.792555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c109e-f356-4922-be02-d724eecc1442 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:18.803202 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c109e-f356-4922-be02-d724eecc1442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-a07c109e-f356-4922-be02-d724eecc1442 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:47:18.811217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:18.812152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:18.813000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:18.817615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 531/1056] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:47:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:47:18.818559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c109e-f356-4922-be02-d724eecc1442 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:18.822065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f492a-f4bf-4f97-ba7e-e015a02b4642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:18.822921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f492a-f4bf-4f97-ba7e-e015a02b4642 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:18.823200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f492a-f4bf-4f97-ba7e-e015a02b4642 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:18.823469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f492a-f4bf-4f97-ba7e-e015a02b4642 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:18.835154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f492a-f4bf-4f97-ba7e-e015a02b4642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-123f492a-f4bf-4f97-ba7e-e015a02b4642 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:47:18.843420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:18.844512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:47:18.845375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:18.849582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 526/1057] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 12:47:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:47:18.850489 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f492a-f4bf-4f97-ba7e-e015a02b4642 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:18.901152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:18.928686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c109e-f356-4922-be02-d724eecc1442 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:18.930165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:18.955468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f492a-f4bf-4f97-ba7e-e015a02b4642 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:18.964952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953ebed-87ec-40e4-bfbd-6837d286973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:18.965623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953ebed-87ec-40e4-bfbd-6837d286973a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:18.965919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953ebed-87ec-40e4-bfbd-6837d286973a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:18.966735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953ebed-87ec-40e4-bfbd-6837d286973a None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:18.967486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953ebed-87ec-40e4-bfbd-6837d286973a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:18.967793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953ebed-87ec-40e4-bfbd-6837d286973a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:18.969161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953ebed-87ec-40e4-bfbd-6837d286973a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:18.977106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 532/1058] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:47:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:18.978324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc8eff-31b4-430d-a4c2-612af01a7a1f None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:18.978614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc8eff-31b4-430d-a4c2-612af01a7a1f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:18.978900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc8eff-31b4-430d-a4c2-612af01a7a1f None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:18.980237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 533/1059] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:18.985280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc8eff-31b4-430d-a4c2-612af01a7a1f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:18.991057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483cc91-96c3-4b3c-a327-f0c394908569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:18.991728 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483cc91-96c3-4b3c-a327-f0c394908569 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:18.992008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483cc91-96c3-4b3c-a327-f0c394908569 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:18.992281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483cc91-96c3-4b3c-a327-f0c394908569 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:18.993022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483cc91-96c3-4b3c-a327-f0c394908569 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:18.993327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483cc91-96c3-4b3c-a327-f0c394908569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:18.994729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483cc91-96c3-4b3c-a327-f0c394908569 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:19.002765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 527/1060] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 12:47:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:19.004122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b234f9-ac4e-4ed0-a7d2-155a2bf4d022 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:19.004416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b234f9-ac4e-4ed0-a7d2-155a2bf4d022 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:19.004696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b234f9-ac4e-4ed0-a7d2-155a2bf4d022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:19.020547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-30b234f9-ac4e-4ed0-a7d2-155a2bf4d022 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:19.021705 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-30b234f9-ac4e-4ed0-a7d2-155a2bf4d022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:19.085668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 528/1061] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:19.086504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b234f9-ac4e-4ed0-a7d2-155a2bf4d022 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:19.099156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc8eff-31b4-430d-a4c2-612af01a7a1f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:19.129893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:19.144173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72827b1-44d5-4606-9881-333ddf0c2f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:19.145193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72827b1-44d5-4606-9881-333ddf0c2f45 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:19.145478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72827b1-44d5-4606-9881-333ddf0c2f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:19.146736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72827b1-44d5-4606-9881-333ddf0c2f45 None admin] PATH_INFO: `/v3/users/1642a7c5d22340da9e899f7b88747f48` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:19.160680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72827b1-44d5-4606-9881-333ddf0c2f45 None admin] RBAC: Authorizing `identity:get_user(user_id=1642a7c5d22340da9e899f7b88747f48)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:19.162129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72827b1-44d5-4606-9881-333ddf0c2f45 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:19.163901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 534/1062] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:18 2022] GET /identity/v3/users/1642a7c5d22340da9e899f7b88747f48 => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:19.164660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72827b1-44d5-4606-9881-333ddf0c2f45 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:19.189993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b234f9-ac4e-4ed0-a7d2-155a2bf4d022 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:19.193341 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:19.228291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467238e-8d73-4cdc-a3e4-4b0649e6b55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:19.229087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467238e-8d73-4cdc-a3e4-4b0649e6b55c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:19.229373 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467238e-8d73-4cdc-a3e4-4b0649e6b55c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:19.229652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467238e-8d73-4cdc-a3e4-4b0649e6b55c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:19.239661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467238e-8d73-4cdc-a3e4-4b0649e6b55c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:19.241108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467238e-8d73-4cdc-a3e4-4b0649e6b55c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-b467238e-8d73-4cdc-a3e4-4b0649e6b55c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:47:19.259787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:19.260673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:19.261585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:47:19.261585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:47:19.261585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:19.261585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:19.261585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 529/1063] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:19.261863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467238e-8d73-4cdc-a3e4-4b0649e6b55c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:19.274915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72827b1-44d5-4606-9881-333ddf0c2f45 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:19.308020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea5b6e-3417-4b6f-a1b7-21c1f7ce2282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:19.308665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea5b6e-3417-4b6f-a1b7-21c1f7ce2282 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:19.308950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea5b6e-3417-4b6f-a1b7-21c1f7ce2282 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:19.309226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea5b6e-3417-4b6f-a1b7-21c1f7ce2282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:19.350169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:19.359912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467238e-8d73-4cdc-a3e4-4b0649e6b55c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:19.375295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea5b6e-3417-4b6f-a1b7-21c1f7ce2282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:19.376752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea5b6e-3417-4b6f-a1b7-21c1f7ce2282 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:19.393369 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05dfd0d-7b4f-4fcd-842a-1bc89842aa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:19.394024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05dfd0d-7b4f-4fcd-842a-1bc89842aa24 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:19.394331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05dfd0d-7b4f-4fcd-842a-1bc89842aa24 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:19.394614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05dfd0d-7b4f-4fcd-842a-1bc89842aa24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:19.400013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:19.463069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05dfd0d-7b4f-4fcd-842a-1bc89842aa24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:19.464431 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05dfd0d-7b4f-4fcd-842a-1bc89842aa24 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:19.479968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 535/1064] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:47:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:47:19.480820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea5b6e-3417-4b6f-a1b7-21c1f7ce2282 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:19.555704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:19.565628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 530/1065] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:47:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:47:19.566485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05dfd0d-7b4f-4fcd-842a-1bc89842aa24 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:19.581719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea5b6e-3417-4b6f-a1b7-21c1f7ce2282 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:19.603591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:19.616348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100b472-a2df-4038-802d-08d55b1d30c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:19.617006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100b472-a2df-4038-802d-08d55b1d30c6 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:19.617291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100b472-a2df-4038-802d-08d55b1d30c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:19.617887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100b472-a2df-4038-802d-08d55b1d30c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:19.618303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100b472-a2df-4038-802d-08d55b1d30c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:19.618616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100b472-a2df-4038-802d-08d55b1d30c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:19.620018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100b472-a2df-4038-802d-08d55b1d30c6 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:19.631506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 536/1066] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:19.632268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100b472-a2df-4038-802d-08d55b1d30c6 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:19.664319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05dfd0d-7b4f-4fcd-842a-1bc89842aa24 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:19.697869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9df39-f767-4f67-9def-8420889804a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:19.698695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9df39-f767-4f67-9def-8420889804a1 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:19.698975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9df39-f767-4f67-9def-8420889804a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:19.699254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9df39-f767-4f67-9def-8420889804a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:19.709153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9df39-f767-4f67-9def-8420889804a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:19.710614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9df39-f767-4f67-9def-8420889804a1 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-44e9df39-f767-4f67-9def-8420889804a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:47:19.728134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:19.729194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:19.730129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:47:19.730129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:47:19.730129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:19.730129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:19.730393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100b472-a2df-4038-802d-08d55b1d30c6 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:19.730483 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 531/1067] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:19.730483 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9df39-f767-4f67-9def-8420889804a1 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:19.763285 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ab96b-a011-4f0a-83eb-365ebc8f7cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:19.764140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ab96b-a011-4f0a-83eb-365ebc8f7cda None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:19.764416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ab96b-a011-4f0a-83eb-365ebc8f7cda None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:19.764684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ab96b-a011-4f0a-83eb-365ebc8f7cda None admin] PATH_INFO: `/v3/projects/84608542a4664c6aaf8a25fb59073b7c` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:19.765850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ab96b-a011-4f0a-83eb-365ebc8f7cda None admin] RBAC: Authorizing `identity:get_project(project_id=84608542a4664c6aaf8a25fb59073b7c)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:19.767294 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ab96b-a011-4f0a-83eb-365ebc8f7cda None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:19.768962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 537/1068] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:47:19 2022] GET /identity/v3/projects/84608542a4664c6aaf8a25fb59073b7c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:19.769695 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ab96b-a011-4f0a-83eb-365ebc8f7cda None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:19.775590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:19.822224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:19.833398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9df39-f767-4f67-9def-8420889804a1 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:19.865963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200d4df-6c9d-42a7-82dd-fd99a3935e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:19.866785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200d4df-6c9d-42a7-82dd-fd99a3935e4b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:19.867068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200d4df-6c9d-42a7-82dd-fd99a3935e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:19.867340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200d4df-6c9d-42a7-82dd-fd99a3935e4b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:19.871788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ab96b-a011-4f0a-83eb-365ebc8f7cda None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:19.874684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200d4df-6c9d-42a7-82dd-fd99a3935e4b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:19.876109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200d4df-6c9d-42a7-82dd-fd99a3935e4b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-7200d4df-6c9d-42a7-82dd-fd99a3935e4b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:47:19.890887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:19.891848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 532/1069] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 12:47:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:19.892823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200d4df-6c9d-42a7-82dd-fd99a3935e4b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:19.907765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7716c8e-43fd-472d-906f-d071028cd67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:19.908432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7716c8e-43fd-472d-906f-d071028cd67b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:19.908718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7716c8e-43fd-472d-906f-d071028cd67b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:19.908994 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7716c8e-43fd-472d-906f-d071028cd67b None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:19.909710 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7716c8e-43fd-472d-906f-d071028cd67b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:19.910017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7716c8e-43fd-472d-906f-d071028cd67b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:19.911443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7716c8e-43fd-472d-906f-d071028cd67b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:19.922973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 538/1070] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:19.923778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7716c8e-43fd-472d-906f-d071028cd67b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:20.012119 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200d4df-6c9d-42a7-82dd-fd99a3935e4b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:20.015709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:20.035450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7716c8e-43fd-472d-906f-d071028cd67b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:20.048195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:20.051090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f21816-47e0-45ae-b922-77beeeb66552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:20.051963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f21816-47e0-45ae-b922-77beeeb66552 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:20.052261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f21816-47e0-45ae-b922-77beeeb66552 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:20.052535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f21816-47e0-45ae-b922-77beeeb66552 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:20.053385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f21816-47e0-45ae-b922-77beeeb66552 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=84608542a4664c6aaf8a25fb59073b7c, user.id=1642a7c5d22340da9e899f7b88747f48) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:20.053704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f21816-47e0-45ae-b922-77beeeb66552 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:20.055237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f21816-47e0-45ae-b922-77beeeb66552 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:20.061875 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 533/1071] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:47:19 2022] GET /identity/v3/role_assignments?user.id=1642a7c5d22340da9e899f7b88747f48&scope.project.id=84608542a4664c6aaf8a25fb59073b7c&role.id=0c473678b281495f86f58b077be04e1b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:20.078879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d04dd6-efa6-4ec5-8bbc-77c16f774846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:20.079582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d04dd6-efa6-4ec5-8bbc-77c16f774846 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:20.079867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d04dd6-efa6-4ec5-8bbc-77c16f774846 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:20.080144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d04dd6-efa6-4ec5-8bbc-77c16f774846 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:20.081021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d04dd6-efa6-4ec5-8bbc-77c16f774846 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:20.081334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d04dd6-efa6-4ec5-8bbc-77c16f774846 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:20.082892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d04dd6-efa6-4ec5-8bbc-77c16f774846 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:20.103709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 539/1072] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 12:47:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:20.110833 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f21816-47e0-45ae-b922-77beeeb66552 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:20.229013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f21816-47e0-45ae-b922-77beeeb66552 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:20.265896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffee80-fb94-4b4d-8536-7db28b329308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:20.266647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffee80-fb94-4b4d-8536-7db28b329308 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:20.266932 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffee80-fb94-4b4d-8536-7db28b329308 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:20.267207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffee80-fb94-4b4d-8536-7db28b329308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:20.277476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffee80-fb94-4b4d-8536-7db28b329308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:20.278998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffee80-fb94-4b4d-8536-7db28b329308 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-baffee80-fb94-4b4d-8536-7db28b329308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:47:20.296723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:20.297658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:20.298629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:47:20.298629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:47:20.298629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:20.298629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:20.298629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 534/1073] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:20.303636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d04dd6-efa6-4ec5-8bbc-77c16f774846 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:20.402638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d04dd6-efa6-4ec5-8bbc-77c16f774846 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:20.436791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbf0c7-dad8-452a-805f-b00525023c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:20.437462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbf0c7-dad8-452a-805f-b00525023c7e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:20.437748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbf0c7-dad8-452a-805f-b00525023c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:20.438025 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbf0c7-dad8-452a-805f-b00525023c7e None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:20.438770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbf0c7-dad8-452a-805f-b00525023c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:20.439080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbf0c7-dad8-452a-805f-b00525023c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:20.440452 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbf0c7-dad8-452a-805f-b00525023c7e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:20.452050 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 540/1074] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:20.457646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffee80-fb94-4b4d-8536-7db28b329308 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:20.478443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:20.554384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffee80-fb94-4b4d-8536-7db28b329308 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:20.586189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa5dcd-156a-4be9-bdba-87e5e735cdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:20.587018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa5dcd-156a-4be9-bdba-87e5e735cdfc None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:20.587298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa5dcd-156a-4be9-bdba-87e5e735cdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:20.587571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa5dcd-156a-4be9-bdba-87e5e735cdfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:20.594553 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa5dcd-156a-4be9-bdba-87e5e735cdfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:20.595982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa5dcd-156a-4be9-bdba-87e5e735cdfc None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-b5fa5dcd-156a-4be9-bdba-87e5e735cdfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:47:20.609811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:20.610769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:20.611852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:47:20.611852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:47:20.611852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:47:20.611852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:47:20.611852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:47:20.611852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:47:20.611852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:47:20.611852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:20.611852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 535/1075] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:47:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:20.616656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbf0c7-dad8-452a-805f-b00525023c7e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:20.644883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:20.713490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbf0c7-dad8-452a-805f-b00525023c7e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:20.746297 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fdd78-6dc0-491e-8788-7ec4d0e045b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:20.746958 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fdd78-6dc0-491e-8788-7ec4d0e045b3 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:20.747237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fdd78-6dc0-491e-8788-7ec4d0e045b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:20.747534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fdd78-6dc0-491e-8788-7ec4d0e045b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:20.748311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fdd78-6dc0-491e-8788-7ec4d0e045b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:20.748620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fdd78-6dc0-491e-8788-7ec4d0e045b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:20.750043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fdd78-6dc0-491e-8788-7ec4d0e045b3 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:20.762631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 541/1076] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:47:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:20.768513 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa5dcd-156a-4be9-bdba-87e5e735cdfc None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:20.847972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:20.866953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa5dcd-156a-4be9-bdba-87e5e735cdfc None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:20.899429 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a3f8a-2b3f-492d-9e55-b0a5ed0efda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:20.900318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a3f8a-2b3f-492d-9e55-b0a5ed0efda8 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:20.900600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a3f8a-2b3f-492d-9e55-b0a5ed0efda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:20.900869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a3f8a-2b3f-492d-9e55-b0a5ed0efda8 None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/05b2edb6fcb7496bb55650080a9da172/roles/b138ce6a3b514198ba4a9bdddb99ab81` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:20.903003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a3f8a-2b3f-492d-9e55-b0a5ed0efda8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=05b2edb6fcb7496bb55650080a9da172, role_id=b138ce6a3b514198ba4a9bdddb99ab81)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:20.905010 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a3f8a-2b3f-492d-9e55-b0a5ed0efda8 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:20.917335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 536/1077] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:47:20 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/05b2edb6fcb7496bb55650080a9da172/roles/b138ce6a3b514198ba4a9bdddb99ab81 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 12:47:22.303049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a17eb-278f-46cd-ad3d-1f83980703c9 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:22.303580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a17eb-278f-46cd-ad3d-1f83980703c9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:22.303580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a17eb-278f-46cd-ad3d-1f83980703c9 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:22.304976 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 542/1078] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:22.310362 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaaed9-5342-4d28-a480-a84a43d211a8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:22.310763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaaed9-5342-4d28-a480-a84a43d211a8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:22.311030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaaed9-5342-4d28-a480-a84a43d211a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:22.328163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9dcaaed9-5342-4d28-a480-a84a43d211a8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:22.329345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9dcaaed9-5342-4d28-a480-a84a43d211a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:22.394327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 537/1079] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:22.416851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a17eb-278f-46cd-ad3d-1f83980703c9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:22.503399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:22.517107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a17eb-278f-46cd-ad3d-1f83980703c9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:22.550428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a3b73-00ae-42cc-a961-20a310f3997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:22.551222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a3b73-00ae-42cc-a961-20a310f3997e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:22.551506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a3b73-00ae-42cc-a961-20a310f3997e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:22.551787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a3b73-00ae-42cc-a961-20a310f3997e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:22.561629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a3b73-00ae-42cc-a961-20a310f3997e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-d18a3b73-00ae-42cc-a961-20a310f3997e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:47:22.569192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:22.570309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:22.571207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 543/1080] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:47:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:22.576954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaaed9-5342-4d28-a480-a84a43d211a8 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:22.644031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:22.675246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaaed9-5342-4d28-a480-a84a43d211a8 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:22.708786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ad122-60f0-4016-94d9-2e70b44fc12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:22.709452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ad122-60f0-4016-94d9-2e70b44fc12c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:22.709738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ad122-60f0-4016-94d9-2e70b44fc12c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:22.710013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ad122-60f0-4016-94d9-2e70b44fc12c None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:22.710790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ad122-60f0-4016-94d9-2e70b44fc12c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:22.711101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ad122-60f0-4016-94d9-2e70b44fc12c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:22.712467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ad122-60f0-4016-94d9-2e70b44fc12c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:22.720864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 538/1081] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:47:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:22.726780 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a3b73-00ae-42cc-a961-20a310f3997e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:22.824195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a3b73-00ae-42cc-a961-20a310f3997e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:22.856698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f15ee-22ed-4e73-a0da-932a8ef34217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:22.858073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f15ee-22ed-4e73-a0da-932a8ef34217 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:22.858392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f15ee-22ed-4e73-a0da-932a8ef34217 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:22.858672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f15ee-22ed-4e73-a0da-932a8ef34217 None admin] PATH_INFO: `/v3/users/1642a7c5d22340da9e899f7b88747f48` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:22.859858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f15ee-22ed-4e73-a0da-932a8ef34217 None admin] RBAC: Authorizing `identity:get_user(user_id=1642a7c5d22340da9e899f7b88747f48)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:22.861302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f15ee-22ed-4e73-a0da-932a8ef34217 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:22.862986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 544/1082] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:22 2022] GET /identity/v3/users/1642a7c5d22340da9e899f7b88747f48 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:22.868608 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ad122-60f0-4016-94d9-2e70b44fc12c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:22.971572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ad122-60f0-4016-94d9-2e70b44fc12c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:22.999625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:23.004839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376ec53-068a-4cfd-9a69-52b9b8670a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:23.005700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ec53-068a-4cfd-9a69-52b9b8670a5b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:23.005982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ec53-068a-4cfd-9a69-52b9b8670a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:23.006258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ec53-068a-4cfd-9a69-52b9b8670a5b None admin] PATH_INFO: `/v3/projects/84608542a4664c6aaf8a25fb59073b7c` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:23.007466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376ec53-068a-4cfd-9a69-52b9b8670a5b None admin] RBAC: Authorizing `identity:get_project(project_id=84608542a4664c6aaf8a25fb59073b7c)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:23.008902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376ec53-068a-4cfd-9a69-52b9b8670a5b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:23.011044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 539/1083] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:47:22 2022] GET /identity/v3/projects/84608542a4664c6aaf8a25fb59073b7c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:23.017383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f15ee-22ed-4e73-a0da-932a8ef34217 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:23.039786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:23.077598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac930cb2-2417-4476-ab71-6713afc5a50f None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:23.078131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac930cb2-2417-4476-ab71-6713afc5a50f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:23.078461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac930cb2-2417-4476-ab71-6713afc5a50f None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:23.079832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 540/1084] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 12:47:23.084651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe230f-37db-4e23-b6c5-4504f57e39a7 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:23.084942 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe230f-37db-4e23-b6c5-4504f57e39a7 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:23.085229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe230f-37db-4e23-b6c5-4504f57e39a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:23.101360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-84fe230f-37db-4e23-b6c5-4504f57e39a7 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:23.102566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-84fe230f-37db-4e23-b6c5-4504f57e39a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:23.121280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f15ee-22ed-4e73-a0da-932a8ef34217 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:23.155999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6d308-07e8-4aec-be02-1ad08e4c72e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:23.156881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d308-07e8-4aec-be02-1ad08e4c72e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:23.157169 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d308-07e8-4aec-be02-1ad08e4c72e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:23.157450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d308-07e8-4aec-be02-1ad08e4c72e2 None admin] PATH_INFO: `/v3/projects/84608542a4664c6aaf8a25fb59073b7c/users/1642a7c5d22340da9e899f7b88747f48/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:23.159567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6d308-07e8-4aec-be02-1ad08e4c72e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=84608542a4664c6aaf8a25fb59073b7c, user_id=1642a7c5d22340da9e899f7b88747f48, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:23.161044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6d308-07e8-4aec-be02-1ad08e4c72e2 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:23.170135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 541/1085] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:47:23.178524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 545/1086] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:47:23 2022] PUT /identity/v3/projects/84608542a4664c6aaf8a25fb59073b7c/users/1642a7c5d22340da9e899f7b88747f48/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:47:23.195746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe230f-37db-4e23-b6c5-4504f57e39a7 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:23.300149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe230f-37db-4e23-b6c5-4504f57e39a7 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:23.329975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:23.336580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386625b-b39e-40be-9176-ff66a3bedb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:23.337424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386625b-b39e-40be-9176-ff66a3bedb5d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:23.337713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386625b-b39e-40be-9176-ff66a3bedb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:23.337994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386625b-b39e-40be-9176-ff66a3bedb5d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:23.350052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386625b-b39e-40be-9176-ff66a3bedb5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-6386625b-b39e-40be-9176-ff66a3bedb5d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:47:23.358425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:23.359481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:23.360410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 542/1087] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 12:47:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:23.365699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6d308-07e8-4aec-be02-1ad08e4c72e2 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:23.446775 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:23.470563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6d308-07e8-4aec-be02-1ad08e4c72e2 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:23.503070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f1795-b07f-4ec9-aa93-64f9cc04d3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:23.503743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1795-b07f-4ec9-aa93-64f9cc04d3ae None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:23.504029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1795-b07f-4ec9-aa93-64f9cc04d3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:23.504313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1795-b07f-4ec9-aa93-64f9cc04d3ae None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:23.505052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f1795-b07f-4ec9-aa93-64f9cc04d3ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:23.505359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f1795-b07f-4ec9-aa93-64f9cc04d3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:23.506770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f1795-b07f-4ec9-aa93-64f9cc04d3ae None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:23.514932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 546/1088] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 12:47:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:23.520500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386625b-b39e-40be-9176-ff66a3bedb5d None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:23.618199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386625b-b39e-40be-9176-ff66a3bedb5d None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:23.650842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08049659-b076-4e13-939b-dafa125d67c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:23.651633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08049659-b076-4e13-939b-dafa125d67c0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:23.651918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08049659-b076-4e13-939b-dafa125d67c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:23.652196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08049659-b076-4e13-939b-dafa125d67c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:23.663021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08049659-b076-4e13-939b-dafa125d67c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:23.664456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08049659-b076-4e13-939b-dafa125d67c0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-08049659-b076-4e13-939b-dafa125d67c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:47:23.681078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:23.682154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:23.683494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:47:23.683494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:47:23.683494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:23.683494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:23.683494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 543/1089] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:23.688387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f1795-b07f-4ec9-aa93-64f9cc04d3ae None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:23.785704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f1795-b07f-4ec9-aa93-64f9cc04d3ae None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:23.818066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463c4bd-4858-499d-93bf-b884da3ae975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:23.818759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463c4bd-4858-499d-93bf-b884da3ae975 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:23.819044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463c4bd-4858-499d-93bf-b884da3ae975 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:23.819319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463c4bd-4858-499d-93bf-b884da3ae975 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:23.820037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463c4bd-4858-499d-93bf-b884da3ae975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:23.820346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463c4bd-4858-499d-93bf-b884da3ae975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:23.821714 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463c4bd-4858-499d-93bf-b884da3ae975 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:23.829108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:23.835192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 547/1090] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:23.840982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08049659-b076-4e13-939b-dafa125d67c0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:23.883291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:23.938882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08049659-b076-4e13-939b-dafa125d67c0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:23.971056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd1a49-fb08-4433-b229-0e6c58980ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:23.971855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd1a49-fb08-4433-b229-0e6c58980ccb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:23.972139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd1a49-fb08-4433-b229-0e6c58980ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:23.972415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd1a49-fb08-4433-b229-0e6c58980ccb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:23.982174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd1a49-fb08-4433-b229-0e6c58980ccb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:23.983650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd1a49-fb08-4433-b229-0e6c58980ccb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-9ffd1a49-fb08-4433-b229-0e6c58980ccb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:47:23.998437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:23.999518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:24.000890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:24.000890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:24.000890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:47:24.000890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:47:24.000890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:47:24.000890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:47:24.000890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:47:24.000890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:47:24.000890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 12:47:24.000890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:24.000890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 544/1091] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 12:47:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:24.005255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463c4bd-4858-499d-93bf-b884da3ae975 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:24.102041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463c4bd-4858-499d-93bf-b884da3ae975 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:24.135281 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c47f59-3f70-4319-a43c-275c6075a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:24.135876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c47f59-3f70-4319-a43c-275c6075a9fc None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:24.136155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c47f59-3f70-4319-a43c-275c6075a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:24.136425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c47f59-3f70-4319-a43c-275c6075a9fc None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:24.137249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c47f59-3f70-4319-a43c-275c6075a9fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:24.137557 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c47f59-3f70-4319-a43c-275c6075a9fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:24.139602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c47f59-3f70-4319-a43c-275c6075a9fc None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:24.156354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 548/1092] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 12:47:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:24.161885 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd1a49-fb08-4433-b229-0e6c58980ccb None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:24.257672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd1a49-fb08-4433-b229-0e6c58980ccb None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:24.259263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:24.293806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc554477-55ac-4368-9ac2-717dffc2b625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:24.294507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc554477-55ac-4368-9ac2-717dffc2b625 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:24.294792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc554477-55ac-4368-9ac2-717dffc2b625 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:24.295068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc554477-55ac-4368-9ac2-717dffc2b625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:24.359903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc554477-55ac-4368-9ac2-717dffc2b625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:24.361354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc554477-55ac-4368-9ac2-717dffc2b625 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:24.459777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 545/1093] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:47:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:47:24.465491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c47f59-3f70-4319-a43c-275c6075a9fc None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:24.521843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:24.563959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c47f59-3f70-4319-a43c-275c6075a9fc None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:24.596462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f63b-a964-46d6-885e-8ec0a5cf7178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:24.597242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f63b-a964-46d6-885e-8ec0a5cf7178 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:24.597522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f63b-a964-46d6-885e-8ec0a5cf7178 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:24.597790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f63b-a964-46d6-885e-8ec0a5cf7178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:24.607836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5f63b-a964-46d6-885e-8ec0a5cf7178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:24.609264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5f63b-a964-46d6-885e-8ec0a5cf7178 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-86e5f63b-a964-46d6-885e-8ec0a5cf7178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:47:24.626464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:24.627543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:24.628804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:47:24.628804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:47:24.628804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:47:24.628804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:24.628804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 549/1094] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:24.633202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc554477-55ac-4368-9ac2-717dffc2b625 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:24.635641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:24.729537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc554477-55ac-4368-9ac2-717dffc2b625 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:24.762882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28b066-0c02-47a5-98b9-01cda40b65e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:24.763544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28b066-0c02-47a5-98b9-01cda40b65e9 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:24.763825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28b066-0c02-47a5-98b9-01cda40b65e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:24.764102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28b066-0c02-47a5-98b9-01cda40b65e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:24.764823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28b066-0c02-47a5-98b9-01cda40b65e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:24.765130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28b066-0c02-47a5-98b9-01cda40b65e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:24.766520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28b066-0c02-47a5-98b9-01cda40b65e9 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:24.777452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 546/1095] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:24.782918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f63b-a964-46d6-885e-8ec0a5cf7178 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:24.880808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f63b-a964-46d6-885e-8ec0a5cf7178 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:24.890494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:24.914972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78bd9e-6126-40dd-a62a-814516341881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:24.915787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78bd9e-6126-40dd-a62a-814516341881 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:24.916070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78bd9e-6126-40dd-a62a-814516341881 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:24.916399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78bd9e-6126-40dd-a62a-814516341881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:24.923412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78bd9e-6126-40dd-a62a-814516341881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:24.924860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78bd9e-6126-40dd-a62a-814516341881 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-1f78bd9e-6126-40dd-a62a-814516341881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:47:24.938816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:24.939719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:24.940656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:47:24.940656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:47:24.940656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:47:24.940656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:47:24.940656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:47:24.940656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:47:24.940656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:47:24.940656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:24.942425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 550/1096] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:47:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:24.945738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28b066-0c02-47a5-98b9-01cda40b65e9 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:25.005605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:25.044742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28b066-0c02-47a5-98b9-01cda40b65e9 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:25.077668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb793743-ad26-4c3f-bbf8-37768bc60339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:25.078348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb793743-ad26-4c3f-bbf8-37768bc60339 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:25.078638 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb793743-ad26-4c3f-bbf8-37768bc60339 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:25.078916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb793743-ad26-4c3f-bbf8-37768bc60339 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:25.079696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb793743-ad26-4c3f-bbf8-37768bc60339 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:25.080007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb793743-ad26-4c3f-bbf8-37768bc60339 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:25.081433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb793743-ad26-4c3f-bbf8-37768bc60339 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:25.093199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 547/1097] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:47:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:25.099201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78bd9e-6126-40dd-a62a-814516341881 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:25.207845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78bd9e-6126-40dd-a62a-814516341881 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:25.242796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fa0ae-95fa-4977-95e7-c49baa4fffb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:25.243450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fa0ae-95fa-4977-95e7-c49baa4fffb8 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:25.243731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fa0ae-95fa-4977-95e7-c49baa4fffb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:25.244002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fa0ae-95fa-4977-95e7-c49baa4fffb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:25.244824 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fa0ae-95fa-4977-95e7-c49baa4fffb8 None admin] RBAC: Adding query filter params (role.id=b138ce6a3b514198ba4a9bdddb99ab81, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=05b2edb6fcb7496bb55650080a9da172) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:25.245134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fa0ae-95fa-4977-95e7-c49baa4fffb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:25.246587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fa0ae-95fa-4977-95e7-c49baa4fffb8 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:25.253961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 551/1098] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:47:25 2022] GET /identity/v3/role_assignments?user.id=05b2edb6fcb7496bb55650080a9da172&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=b138ce6a3b514198ba4a9bdddb99ab81 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:25.378855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bc7f7-c2b3-4268-a16e-5981a9167000 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:25.379404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bc7f7-c2b3-4268-a16e-5981a9167000 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:25.379404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bc7f7-c2b3-4268-a16e-5981a9167000 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:25.380819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 548/1099] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:25.386972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3626ebd-04ec-494e-880a-a11eee78751a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:25.387270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3626ebd-04ec-494e-880a-a11eee78751a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:25.387551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3626ebd-04ec-494e-880a-a11eee78751a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:25.405111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f3626ebd-04ec-494e-880a-a11eee78751a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:25.406363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f3626ebd-04ec-494e-880a-a11eee78751a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:25.433857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:25.477576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 552/1100] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:25.501481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bc7f7-c2b3-4268-a16e-5981a9167000 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:25.624571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bc7f7-c2b3-4268-a16e-5981a9167000 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:25.648778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:25.661629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f889764-53fa-4c24-9ee5-44c74f38ad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:25.662454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f889764-53fa-4c24-9ee5-44c74f38ad72 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:25.662743 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f889764-53fa-4c24-9ee5-44c74f38ad72 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:25.663016 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f889764-53fa-4c24-9ee5-44c74f38ad72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:25.673079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f889764-53fa-4c24-9ee5-44c74f38ad72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-1f889764-53fa-4c24-9ee5-44c74f38ad72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:47:25.680600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:25.681680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:25.683120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 549/1101] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:47:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:25.688328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3626ebd-04ec-494e-880a-a11eee78751a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:25.792032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3626ebd-04ec-494e-880a-a11eee78751a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:25.827320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414ab70-21fd-422f-96eb-14d64c6be92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:25.827993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ab70-21fd-422f-96eb-14d64c6be92d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:25.828280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ab70-21fd-422f-96eb-14d64c6be92d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:25.828564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ab70-21fd-422f-96eb-14d64c6be92d None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:25.829323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414ab70-21fd-422f-96eb-14d64c6be92d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:25.829628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414ab70-21fd-422f-96eb-14d64c6be92d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:25.831029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414ab70-21fd-422f-96eb-14d64c6be92d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:25.839612 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 553/1102] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:47:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:25.846159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f889764-53fa-4c24-9ee5-44c74f38ad72 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:25.949886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f889764-53fa-4c24-9ee5-44c74f38ad72 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:25.984405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e81b7-f35d-4068-8214-d3a4f7700b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:25.985290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e81b7-f35d-4068-8214-d3a4f7700b11 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:25.985568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e81b7-f35d-4068-8214-d3a4f7700b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:25.985840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e81b7-f35d-4068-8214-d3a4f7700b11 None admin] PATH_INFO: `/v3/users/1642a7c5d22340da9e899f7b88747f48` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:25.987689 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e81b7-f35d-4068-8214-d3a4f7700b11 None admin] RBAC: Authorizing `identity:get_user(user_id=1642a7c5d22340da9e899f7b88747f48)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:25.989124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e81b7-f35d-4068-8214-d3a4f7700b11 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:25.990801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 550/1103] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:25 2022] GET /identity/v3/users/1642a7c5d22340da9e899f7b88747f48 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:26.000919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414ab70-21fd-422f-96eb-14d64c6be92d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:26.029561 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:26.104804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414ab70-21fd-422f-96eb-14d64c6be92d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:26.137524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890768c6-2427-459d-ab78-fa913d284535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:26.138718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890768c6-2427-459d-ab78-fa913d284535 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:26.139002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890768c6-2427-459d-ab78-fa913d284535 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:26.139274 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890768c6-2427-459d-ab78-fa913d284535 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:26.203516 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890768c6-2427-459d-ab78-fa913d284535 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:26.204916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890768c6-2427-459d-ab78-fa913d284535 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:26.227828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:26.299977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 554/1104] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:47:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:47:26.305840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e81b7-f35d-4068-8214-d3a4f7700b11 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:26.354133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:26.402366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e81b7-f35d-4068-8214-d3a4f7700b11 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:26.435260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ff538-8e48-4c36-92bb-7b967b403afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:26.436111 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ff538-8e48-4c36-92bb-7b967b403afd None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:26.436396 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ff538-8e48-4c36-92bb-7b967b403afd None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:26.436673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ff538-8e48-4c36-92bb-7b967b403afd None admin] PATH_INFO: `/v3/projects/84608542a4664c6aaf8a25fb59073b7c` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:26.437835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ff538-8e48-4c36-92bb-7b967b403afd None admin] RBAC: Authorizing `identity:get_project(project_id=84608542a4664c6aaf8a25fb59073b7c)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:26.439281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ff538-8e48-4c36-92bb-7b967b403afd None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:26.440957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 551/1105] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:47:26 2022] GET /identity/v3/projects/84608542a4664c6aaf8a25fb59073b7c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:26.446943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890768c6-2427-459d-ab78-fa913d284535 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:26.541759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890768c6-2427-459d-ab78-fa913d284535 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:26.564438 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:26.575079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70ed5c-b574-442e-a259-91d55e380efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:26.575748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70ed5c-b574-442e-a259-91d55e380efc None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:26.576030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70ed5c-b574-442e-a259-91d55e380efc None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:26.576303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70ed5c-b574-442e-a259-91d55e380efc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:26.577123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70ed5c-b574-442e-a259-91d55e380efc None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=84608542a4664c6aaf8a25fb59073b7c, user.id=1642a7c5d22340da9e899f7b88747f48) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:26.577429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70ed5c-b574-442e-a259-91d55e380efc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:26.578888 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70ed5c-b574-442e-a259-91d55e380efc None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:26.584486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 555/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:47:26 2022] GET /identity/v3/role_assignments?user.id=1642a7c5d22340da9e899f7b88747f48&scope.project.id=84608542a4664c6aaf8a25fb59073b7c&role.id=0c473678b281495f86f58b077be04e1b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:28.768243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090ca5a-f997-427f-8a18-2c3901b144f3 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:28.768746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090ca5a-f997-427f-8a18-2c3901b144f3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:28.768746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090ca5a-f997-427f-8a18-2c3901b144f3 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:28.770094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 552/1107] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:28.775286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c99ad3-7cfd-49f5-abda-fbf9b41bccb5 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:28.775664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c99ad3-7cfd-49f5-abda-fbf9b41bccb5 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:28.775831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c99ad3-7cfd-49f5-abda-fbf9b41bccb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:28.792462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c7c99ad3-7cfd-49f5-abda-fbf9b41bccb5 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:28.793623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c7c99ad3-7cfd-49f5-abda-fbf9b41bccb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:28.857720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 556/1108] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:28.881487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090ca5a-f997-427f-8a18-2c3901b144f3 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:28.980650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090ca5a-f997-427f-8a18-2c3901b144f3 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:28.997301 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:29.013963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a40eb2-3a1d-46d2-a967-ba4a6956f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:29.014784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40eb2-3a1d-46d2-a967-ba4a6956f601 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:29.015071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40eb2-3a1d-46d2-a967-ba4a6956f601 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:29.015345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40eb2-3a1d-46d2-a967-ba4a6956f601 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:29.016519 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a40eb2-3a1d-46d2-a967-ba4a6956f601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:29.017946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a40eb2-3a1d-46d2-a967-ba4a6956f601 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:29.019625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 553/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:29.025225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c99ad3-7cfd-49f5-abda-fbf9b41bccb5 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:29.120243 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c99ad3-7cfd-49f5-abda-fbf9b41bccb5 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:29.137486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:29.154610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b17459-c3a0-4aff-8ac7-fdbb494fc88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:29.155260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b17459-c3a0-4aff-8ac7-fdbb494fc88c None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:29.155539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b17459-c3a0-4aff-8ac7-fdbb494fc88c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:29.155812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b17459-c3a0-4aff-8ac7-fdbb494fc88c None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:29.156504 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b17459-c3a0-4aff-8ac7-fdbb494fc88c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:29.157925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b17459-c3a0-4aff-8ac7-fdbb494fc88c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:29.178168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 557/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:47:29 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:47:31.375007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca2868-61ef-4585-8864-9fe7336608cb None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:31.375560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca2868-61ef-4585-8864-9fe7336608cb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:31.375560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca2868-61ef-4585-8864-9fe7336608cb None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:31.376922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 554/1111] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:31.382205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcfd54-834b-4388-8fc3-f93a0962bece None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:31.382640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcfd54-834b-4388-8fc3-f93a0962bece None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:31.382808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcfd54-834b-4388-8fc3-f93a0962bece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:31.400029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-57bcfd54-834b-4388-8fc3-f93a0962bece None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:31.401211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-57bcfd54-834b-4388-8fc3-f93a0962bece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:31.466576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 558/1112] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:31.489476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca2868-61ef-4585-8864-9fe7336608cb None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:31.588745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca2868-61ef-4585-8864-9fe7336608cb None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:31.620923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485120f-b4ee-46b9-af19-1e98a78adbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:31.621777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485120f-b4ee-46b9-af19-1e98a78adbfd None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:31.622095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485120f-b4ee-46b9-af19-1e98a78adbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:31.622399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485120f-b4ee-46b9-af19-1e98a78adbfd None admin] PATH_INFO: `/v3/roles/c7b7d3b4dd944936851dee41ead5dc99` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:31.623863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485120f-b4ee-46b9-af19-1e98a78adbfd None admin] RBAC: Authorizing `identity:get_role(role_id=c7b7d3b4dd944936851dee41ead5dc99)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:31.625272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485120f-b4ee-46b9-af19-1e98a78adbfd None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:31.626627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 555/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:31 2022] GET /identity/v3/roles/c7b7d3b4dd944936851dee41ead5dc99 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:31.632478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcfd54-834b-4388-8fc3-f93a0962bece None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:31.727907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcfd54-834b-4388-8fc3-f93a0962bece None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:31.730116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:31.761122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc8431-2e3f-40bd-af82-a8e6016abe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:31.761978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc8431-2e3f-40bd-af82-a8e6016abe24 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:31.762259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc8431-2e3f-40bd-af82-a8e6016abe24 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:31.763133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc8431-2e3f-40bd-af82-a8e6016abe24 None admin] PATH_INFO: `/v3/users/d1c65fada8ed4496aa9429274e44a11b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:31.775539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc8431-2e3f-40bd-af82-a8e6016abe24 None admin] RBAC: Authorizing `identity:get_user(user_id=d1c65fada8ed4496aa9429274e44a11b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:31.776972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc8431-2e3f-40bd-af82-a8e6016abe24 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:31.778597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 559/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:31 2022] GET /identity/v3/users/d1c65fada8ed4496aa9429274e44a11b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:31.784210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485120f-b4ee-46b9-af19-1e98a78adbfd None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:31.837455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:31.879321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485120f-b4ee-46b9-af19-1e98a78adbfd None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:31.911012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034af72-a91f-494d-834b-3d91c90c8fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:31.911642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034af72-a91f-494d-834b-3d91c90c8fab None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:31.911921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034af72-a91f-494d-834b-3d91c90c8fab None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:31.912192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034af72-a91f-494d-834b-3d91c90c8fab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:31.976387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034af72-a91f-494d-834b-3d91c90c8fab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:31.977826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034af72-a91f-494d-834b-3d91c90c8fab None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:32.029115 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:32.071449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 556/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:47:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:47:32.077744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc8431-2e3f-40bd-af82-a8e6016abe24 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:32.171808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc8431-2e3f-40bd-af82-a8e6016abe24 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:32.203964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c972ed-b5ca-471d-b20c-34a2abd77b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:32.204801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c972ed-b5ca-471d-b20c-34a2abd77b9d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:32.205084 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c972ed-b5ca-471d-b20c-34a2abd77b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:32.205358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c972ed-b5ca-471d-b20c-34a2abd77b9d None admin] PATH_INFO: `/v3/projects/84608542a4664c6aaf8a25fb59073b7c` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:32.206536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c972ed-b5ca-471d-b20c-34a2abd77b9d None admin] RBAC: Authorizing `identity:get_project(project_id=84608542a4664c6aaf8a25fb59073b7c)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:32.207956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c972ed-b5ca-471d-b20c-34a2abd77b9d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:32.209594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 560/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:47:32 2022] GET /identity/v3/projects/84608542a4664c6aaf8a25fb59073b7c => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:32.215709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034af72-a91f-494d-834b-3d91c90c8fab None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:32.310204 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034af72-a91f-494d-834b-3d91c90c8fab None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:32.343227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab64be7-8a95-443e-a57c-17c5d793a0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:32.343872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab64be7-8a95-443e-a57c-17c5d793a0bd None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:32.344156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab64be7-8a95-443e-a57c-17c5d793a0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:32.344430 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab64be7-8a95-443e-a57c-17c5d793a0bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:32.345248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab64be7-8a95-443e-a57c-17c5d793a0bd None admin] RBAC: Adding query filter params (role.id=c7b7d3b4dd944936851dee41ead5dc99, scope.project.id=84608542a4664c6aaf8a25fb59073b7c, user.id=d1c65fada8ed4496aa9429274e44a11b) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:32.345560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab64be7-8a95-443e-a57c-17c5d793a0bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:32.347025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab64be7-8a95-443e-a57c-17c5d793a0bd None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:32.352302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 557/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:47:32 2022] GET /identity/v3/role_assignments?user.id=d1c65fada8ed4496aa9429274e44a11b&scope.project.id=84608542a4664c6aaf8a25fb59073b7c&role.id=c7b7d3b4dd944936851dee41ead5dc99 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:34.505640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c3fdb-a954-4443-b67e-bbaa3d42a796 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:34.506134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c3fdb-a954-4443-b67e-bbaa3d42a796 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:34.506355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c3fdb-a954-4443-b67e-bbaa3d42a796 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:34.507587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 561/1118] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:34.513641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826be08-e320-4ea9-a9d8-cee112c5aef3 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:34.514041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826be08-e320-4ea9-a9d8-cee112c5aef3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:34.514193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826be08-e320-4ea9-a9d8-cee112c5aef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:34.531056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e826be08-e320-4ea9-a9d8-cee112c5aef3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:34.532227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e826be08-e320-4ea9-a9d8-cee112c5aef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:34.545015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:34.604102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 558/1119] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:34.626126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c3fdb-a954-4443-b67e-bbaa3d42a796 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:34.656960 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:34.726087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c3fdb-a954-4443-b67e-bbaa3d42a796 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:34.758701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96a357-446b-4948-9ffa-d021151b06b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:34.759795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96a357-446b-4948-9ffa-d021151b06b0 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:34.760079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96a357-446b-4948-9ffa-d021151b06b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:34.760358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96a357-446b-4948-9ffa-d021151b06b0 None admin] PATH_INFO: `/v3/roles/c7b7d3b4dd944936851dee41ead5dc99` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:34.761822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96a357-446b-4948-9ffa-d021151b06b0 None admin] RBAC: Authorizing `identity:get_role(role_id=c7b7d3b4dd944936851dee41ead5dc99)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:34.763791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96a357-446b-4948-9ffa-d021151b06b0 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:34.765129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 562/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:34 2022] GET /identity/v3/roles/c7b7d3b4dd944936851dee41ead5dc99 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:34.770731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826be08-e320-4ea9-a9d8-cee112c5aef3 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:34.865074 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826be08-e320-4ea9-a9d8-cee112c5aef3 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:34.898018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65d374-56d8-4846-addf-e33681a979af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:34.898910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65d374-56d8-4846-addf-e33681a979af None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:34.899199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65d374-56d8-4846-addf-e33681a979af None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:34.899476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65d374-56d8-4846-addf-e33681a979af None admin] PATH_INFO: `/v3/users/d1c65fada8ed4496aa9429274e44a11b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:34.900648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65d374-56d8-4846-addf-e33681a979af None admin] RBAC: Authorizing `identity:get_user(user_id=d1c65fada8ed4496aa9429274e44a11b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:34.902066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65d374-56d8-4846-addf-e33681a979af None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:34.904214 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 559/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:34 2022] GET /identity/v3/users/d1c65fada8ed4496aa9429274e44a11b => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:34.909987 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96a357-446b-4948-9ffa-d021151b06b0 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:35.003086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:35.007253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96a357-446b-4948-9ffa-d021151b06b0 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:35.040248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c219d-2f88-4004-9530-abafd315ddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:35.041085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c219d-2f88-4004-9530-abafd315ddd7 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:35.041367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c219d-2f88-4004-9530-abafd315ddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:35.041639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c219d-2f88-4004-9530-abafd315ddd7 None admin] PATH_INFO: `/v3/projects/84608542a4664c6aaf8a25fb59073b7c` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:35.042815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c219d-2f88-4004-9530-abafd315ddd7 None admin] RBAC: Authorizing `identity:get_project(project_id=84608542a4664c6aaf8a25fb59073b7c)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:35.044797 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c219d-2f88-4004-9530-abafd315ddd7 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:35.046453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 563/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:47:34 2022] GET /identity/v3/projects/84608542a4664c6aaf8a25fb59073b7c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:35.052124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65d374-56d8-4846-addf-e33681a979af None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:35.069487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:35.147824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65d374-56d8-4846-addf-e33681a979af None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:35.180819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c146676-6923-49e0-ab96-2aecda9a4e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:35.181641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c146676-6923-49e0-ab96-2aecda9a4e3a None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:35.181927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c146676-6923-49e0-ab96-2aecda9a4e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:35.182201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c146676-6923-49e0-ab96-2aecda9a4e3a None admin] PATH_INFO: `/v3/projects/84608542a4664c6aaf8a25fb59073b7c/users/d1c65fada8ed4496aa9429274e44a11b/roles/c7b7d3b4dd944936851dee41ead5dc99` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:35.184415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c146676-6923-49e0-ab96-2aecda9a4e3a None admin] RBAC: Authorizing `identity:create_grant(project_id=84608542a4664c6aaf8a25fb59073b7c, user_id=d1c65fada8ed4496aa9429274e44a11b, role_id=c7b7d3b4dd944936851dee41ead5dc99)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:35.185823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c146676-6923-49e0-ab96-2aecda9a4e3a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:35.198931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 560/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:47:35 2022] PUT /identity/v3/projects/84608542a4664c6aaf8a25fb59073b7c/users/d1c65fada8ed4496aa9429274e44a11b/roles/c7b7d3b4dd944936851dee41ead5dc99 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:47:37.362130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90621b2f-0845-4943-bd89-85b7d96cadac None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:37.362691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90621b2f-0845-4943-bd89-85b7d96cadac None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:37.362691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90621b2f-0845-4943-bd89-85b7d96cadac None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:37.364066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 564/1124] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:37.369317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf009-9a88-41be-8ef4-56fc9828d767 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:37.369688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf009-9a88-41be-8ef4-56fc9828d767 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:37.369877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf009-9a88-41be-8ef4-56fc9828d767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:37.386571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e33bf009-9a88-41be-8ef4-56fc9828d767 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:37.388274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e33bf009-9a88-41be-8ef4-56fc9828d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:37.455895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 561/1125] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:37.479887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90621b2f-0845-4943-bd89-85b7d96cadac None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:37.581570 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90621b2f-0845-4943-bd89-85b7d96cadac None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:37.614133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48038988-f492-4763-a9b6-81575634f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:37.614951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48038988-f492-4763-a9b6-81575634f36e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:37.615236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48038988-f492-4763-a9b6-81575634f36e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:37.615511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48038988-f492-4763-a9b6-81575634f36e None admin] PATH_INFO: `/v3/roles/c7b7d3b4dd944936851dee41ead5dc99` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:37.616969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48038988-f492-4763-a9b6-81575634f36e None admin] RBAC: Authorizing `identity:get_role(role_id=c7b7d3b4dd944936851dee41ead5dc99)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:37.618367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48038988-f492-4763-a9b6-81575634f36e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:37.619718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 565/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:37 2022] GET /identity/v3/roles/c7b7d3b4dd944936851dee41ead5dc99 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:37.625604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bf009-9a88-41be-8ef4-56fc9828d767 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:37.633430 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:37.722018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bf009-9a88-41be-8ef4-56fc9828d767 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:37.754187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bb898-76ed-49e1-b8ae-83064ca7558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:37.755070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bb898-76ed-49e1-b8ae-83064ca7558b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:37.755353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bb898-76ed-49e1-b8ae-83064ca7558b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:37.755627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bb898-76ed-49e1-b8ae-83064ca7558b None admin] PATH_INFO: `/v3/users/d1c65fada8ed4496aa9429274e44a11b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:37.756795 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949bb898-76ed-49e1-b8ae-83064ca7558b None admin] RBAC: Authorizing `identity:get_user(user_id=d1c65fada8ed4496aa9429274e44a11b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:37.758221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949bb898-76ed-49e1-b8ae-83064ca7558b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:37.760105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 562/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:37 2022] GET /identity/v3/users/d1c65fada8ed4496aa9429274e44a11b => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:37.766270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48038988-f492-4763-a9b6-81575634f36e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:37.799749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:37.863703 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48038988-f492-4763-a9b6-81575634f36e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:37.896217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148e938-d4e3-4254-b5cb-c17e22e98158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:37.896795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148e938-d4e3-4254-b5cb-c17e22e98158 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:37.897081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148e938-d4e3-4254-b5cb-c17e22e98158 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:37.897359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148e938-d4e3-4254-b5cb-c17e22e98158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:37.962816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148e938-d4e3-4254-b5cb-c17e22e98158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:37.964257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148e938-d4e3-4254-b5cb-c17e22e98158 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:37.994912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:38.059999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 566/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:47:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:47:38.065647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bb898-76ed-49e1-b8ae-83064ca7558b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:38.137386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:38.161294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bb898-76ed-49e1-b8ae-83064ca7558b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:38.194616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab9cd8-ca51-4fd9-8dab-15acb47a931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:38.195462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab9cd8-ca51-4fd9-8dab-15acb47a931d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:38.195756 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab9cd8-ca51-4fd9-8dab-15acb47a931d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:38.196014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab9cd8-ca51-4fd9-8dab-15acb47a931d None admin] PATH_INFO: `/v3/projects/84608542a4664c6aaf8a25fb59073b7c` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:38.197179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab9cd8-ca51-4fd9-8dab-15acb47a931d None admin] RBAC: Authorizing `identity:get_project(project_id=84608542a4664c6aaf8a25fb59073b7c)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:38.198618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab9cd8-ca51-4fd9-8dab-15acb47a931d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:38.200274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 563/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:47:38 2022] GET /identity/v3/projects/84608542a4664c6aaf8a25fb59073b7c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:38.206107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148e938-d4e3-4254-b5cb-c17e22e98158 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:38.301947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148e938-d4e3-4254-b5cb-c17e22e98158 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:38.334831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a42fcd-5e22-4455-9acb-5fe804c9067e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:38.335470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42fcd-5e22-4455-9acb-5fe804c9067e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:38.335758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42fcd-5e22-4455-9acb-5fe804c9067e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:38.336036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42fcd-5e22-4455-9acb-5fe804c9067e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:38.336888 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a42fcd-5e22-4455-9acb-5fe804c9067e None admin] RBAC: Adding query filter params (role.id=c7b7d3b4dd944936851dee41ead5dc99, scope.project.id=84608542a4664c6aaf8a25fb59073b7c, user.id=d1c65fada8ed4496aa9429274e44a11b) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:38.337203 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a42fcd-5e22-4455-9acb-5fe804c9067e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:38.338652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a42fcd-5e22-4455-9acb-5fe804c9067e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:38.340967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:38.345344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 567/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:47:38 2022] GET /identity/v3/role_assignments?user.id=d1c65fada8ed4496aa9429274e44a11b&scope.project.id=84608542a4664c6aaf8a25fb59073b7c&role.id=c7b7d3b4dd944936851dee41ead5dc99 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:40.532252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ea59b-8e0c-4b38-a8d0-858cce63dd33 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:40.532765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ea59b-8e0c-4b38-a8d0-858cce63dd33 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:40.532877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ea59b-8e0c-4b38-a8d0-858cce63dd33 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:40.534150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 564/1131] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:40.539301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b491878-3bdf-448b-b9f5-da24b130e264 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:40.539669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b491878-3bdf-448b-b9f5-da24b130e264 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:40.539868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b491878-3bdf-448b-b9f5-da24b130e264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:40.556326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-2b491878-3bdf-448b-b9f5-da24b130e264 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:40.557503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2b491878-3bdf-448b-b9f5-da24b130e264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:40.624947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 568/1132] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:40.648293 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ea59b-8e0c-4b38-a8d0-858cce63dd33 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:40.744343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ea59b-8e0c-4b38-a8d0-858cce63dd33 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:40.775770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3329eab-3923-4a28-8547-38500bb3a91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:40.776542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329eab-3923-4a28-8547-38500bb3a91a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:40.776823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329eab-3923-4a28-8547-38500bb3a91a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:40.777092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329eab-3923-4a28-8547-38500bb3a91a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:40.778756 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3329eab-3923-4a28-8547-38500bb3a91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:40.780163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3329eab-3923-4a28-8547-38500bb3a91a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:40.781719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 565/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:40.787268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b491878-3bdf-448b-b9f5-da24b130e264 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:40.882627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b491878-3bdf-448b-b9f5-da24b130e264 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:40.908984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:40.917883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27694d11-23f1-4a70-b736-9a135ae0d8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:40.919169 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27694d11-23f1-4a70-b736-9a135ae0d8b9 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:40.919457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27694d11-23f1-4a70-b736-9a135ae0d8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:40.919730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27694d11-23f1-4a70-b736-9a135ae0d8b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:40.920450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27694d11-23f1-4a70-b736-9a135ae0d8b9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:40.921896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27694d11-23f1-4a70-b736-9a135ae0d8b9 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:40.950476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 569/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:47:40 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:47:43.134261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d2ef3-66a4-45a8-8fdd-b52929bd756a None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:43.134851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d2ef3-66a4-45a8-8fdd-b52929bd756a None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:43.134851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d2ef3-66a4-45a8-8fdd-b52929bd756a None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:43.136212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 566/1135] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:43.141562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f23bb-4e1d-4253-9858-8174568784ed None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:43.141959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f23bb-4e1d-4253-9858-8174568784ed None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:43.142126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f23bb-4e1d-4253-9858-8174568784ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:43.159103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-610f23bb-4e1d-4253-9858-8174568784ed None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:43.160279 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-610f23bb-4e1d-4253-9858-8174568784ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:43.226073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 570/1136] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:43.250566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d2ef3-66a4-45a8-8fdd-b52929bd756a None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:43.262748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:43.351250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d2ef3-66a4-45a8-8fdd-b52929bd756a None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:43.383338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5145ea-7d76-49fb-bffb-5726ac7e2d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:43.384445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5145ea-7d76-49fb-bffb-5726ac7e2d7c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:43.384665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5145ea-7d76-49fb-bffb-5726ac7e2d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:43.384937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5145ea-7d76-49fb-bffb-5726ac7e2d7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:43.386107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5145ea-7d76-49fb-bffb-5726ac7e2d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:43.388108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5145ea-7d76-49fb-bffb-5726ac7e2d7c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:43.389718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 567/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:47:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:43.395436 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f23bb-4e1d-4253-9858-8174568784ed None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:43.490039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f23bb-4e1d-4253-9858-8174568784ed None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:43.500182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:43.523654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7f1b5-4e18-4e9d-80bf-432de15f3650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:43.524314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7f1b5-4e18-4e9d-80bf-432de15f3650 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:43.524596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7f1b5-4e18-4e9d-80bf-432de15f3650 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:43.524870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7f1b5-4e18-4e9d-80bf-432de15f3650 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:43.525580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7f1b5-4e18-4e9d-80bf-432de15f3650 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:43.527041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7f1b5-4e18-4e9d-80bf-432de15f3650 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:43.547345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 571/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:47:43 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:47:45.765621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d0c9e-c17a-4b10-98d4-32b68339db22 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:45.766127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d0c9e-c17a-4b10-98d4-32b68339db22 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:45.766127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d0c9e-c17a-4b10-98d4-32b68339db22 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:45.767472 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 568/1139] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:45.772691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c29913-5cc1-4f55-b1f1-c0a85df7d6ad None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:45.773098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c29913-5cc1-4f55-b1f1-c0a85df7d6ad None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:45.773280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c29913-5cc1-4f55-b1f1-c0a85df7d6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:45.792049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-08c29913-5cc1-4f55-b1f1-c0a85df7d6ad None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:45.793244 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-08c29913-5cc1-4f55-b1f1-c0a85df7d6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:45.860440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 572/1140] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:45.883316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d0c9e-c17a-4b10-98d4-32b68339db22 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:45.959592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:45.981008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d0c9e-c17a-4b10-98d4-32b68339db22 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:46.014419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb6360-99a9-40d4-821d-6a34de0dbd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:46.015209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb6360-99a9-40d4-821d-6a34de0dbd97 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:46.015497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb6360-99a9-40d4-821d-6a34de0dbd97 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:46.015822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb6360-99a9-40d4-821d-6a34de0dbd97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:46.025596 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb6360-99a9-40d4-821d-6a34de0dbd97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-badb6360-99a9-40d4-821d-6a34de0dbd97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:47:46.033203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:46.034332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:46.035403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 569/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:47:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:46.040167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c29913-5cc1-4f55-b1f1-c0a85df7d6ad None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:46.129335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:46.135890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c29913-5cc1-4f55-b1f1-c0a85df7d6ad None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:46.168581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60582110-206f-40be-8859-2dcb7d4a9dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:46.169229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60582110-206f-40be-8859-2dcb7d4a9dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:46.169508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60582110-206f-40be-8859-2dcb7d4a9dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:46.169780 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60582110-206f-40be-8859-2dcb7d4a9dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:46.170543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60582110-206f-40be-8859-2dcb7d4a9dd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:46.170849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60582110-206f-40be-8859-2dcb7d4a9dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:46.172281 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60582110-206f-40be-8859-2dcb7d4a9dd0 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:46.180945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 573/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:47:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:46.187454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb6360-99a9-40d4-821d-6a34de0dbd97 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:46.282760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb6360-99a9-40d4-821d-6a34de0dbd97 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:46.314298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966cfa9-8bfc-4da8-83fd-c7ca3e8019bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:46.315157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966cfa9-8bfc-4da8-83fd-c7ca3e8019bd None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:46.315438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966cfa9-8bfc-4da8-83fd-c7ca3e8019bd None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:46.315711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966cfa9-8bfc-4da8-83fd-c7ca3e8019bd None admin] PATH_INFO: `/v3/users/d8ce622e5c3c454b830534c208bae585` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:46.323652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:46.330247 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966cfa9-8bfc-4da8-83fd-c7ca3e8019bd None admin] RBAC: Authorizing `identity:get_user(user_id=d8ce622e5c3c454b830534c208bae585)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:46.331693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966cfa9-8bfc-4da8-83fd-c7ca3e8019bd None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:46.333292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 570/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:46 2022] GET /identity/v3/users/d8ce622e5c3c454b830534c208bae585 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:46.338911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60582110-206f-40be-8859-2dcb7d4a9dd0 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:46.435855 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60582110-206f-40be-8859-2dcb7d4a9dd0 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:46.467389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf13c7c-9184-4792-ab62-b0e883399bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:46.468024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf13c7c-9184-4792-ab62-b0e883399bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:46.468308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf13c7c-9184-4792-ab62-b0e883399bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:46.468584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf13c7c-9184-4792-ab62-b0e883399bd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:46.494212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:46.533743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf13c7c-9184-4792-ab62-b0e883399bd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:46.535161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf13c7c-9184-4792-ab62-b0e883399bd5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:46.646935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 574/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:47:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:47:46.653428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966cfa9-8bfc-4da8-83fd-c7ca3e8019bd None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:46.748205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966cfa9-8bfc-4da8-83fd-c7ca3e8019bd None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:46.780766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351733b8-a481-4b3c-9201-005510553e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:46.781615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351733b8-a481-4b3c-9201-005510553e6a None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:46.781900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351733b8-a481-4b3c-9201-005510553e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:46.782175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351733b8-a481-4b3c-9201-005510553e6a None admin] PATH_INFO: `/v3/projects/071cbe4e324e4c45b4142e58e6bb3b14` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:46.783384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351733b8-a481-4b3c-9201-005510553e6a None admin] RBAC: Authorizing `identity:get_project(project_id=071cbe4e324e4c45b4142e58e6bb3b14)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:46.784851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351733b8-a481-4b3c-9201-005510553e6a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:46.786509 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 571/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:47:46 2022] GET /identity/v3/projects/071cbe4e324e4c45b4142e58e6bb3b14 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:46.792355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf13c7c-9184-4792-ab62-b0e883399bd5 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:46.847371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:46.888194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf13c7c-9184-4792-ab62-b0e883399bd5 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:46.919882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf944f-916f-4a24-9fa8-6af04a2a8319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:46.920528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf944f-916f-4a24-9fa8-6af04a2a8319 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:46.920810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf944f-916f-4a24-9fa8-6af04a2a8319 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:46.921082 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf944f-916f-4a24-9fa8-6af04a2a8319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:46.921900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf944f-916f-4a24-9fa8-6af04a2a8319 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=071cbe4e324e4c45b4142e58e6bb3b14, user.id=d8ce622e5c3c454b830534c208bae585) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:46.922208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf944f-916f-4a24-9fa8-6af04a2a8319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:46.923649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf944f-916f-4a24-9fa8-6af04a2a8319 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:46.928848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 575/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:47:46 2022] GET /identity/v3/role_assignments?user.id=d8ce622e5c3c454b830534c208bae585&scope.project.id=071cbe4e324e4c45b4142e58e6bb3b14&role.id=0c473678b281495f86f58b077be04e1b => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:47:49.082526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dfd86-7565-433a-bd32-405cf9d567e4 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:49.083060 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dfd86-7565-433a-bd32-405cf9d567e4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:49.083060 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dfd86-7565-433a-bd32-405cf9d567e4 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:49.084445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 572/1147] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:49.089539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb59f08-1a5f-4869-b89a-9b7e2495d635 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:49.090500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb59f08-1a5f-4869-b89a-9b7e2495d635 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:49.090619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb59f08-1a5f-4869-b89a-9b7e2495d635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:49.107597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8cb59f08-1a5f-4869-b89a-9b7e2495d635 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:49.108769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8cb59f08-1a5f-4869-b89a-9b7e2495d635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:49.172697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 576/1148] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:49.194974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dfd86-7565-433a-bd32-405cf9d567e4 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:49.256239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:49.294769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dfd86-7565-433a-bd32-405cf9d567e4 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:49.327331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721deec3-834c-43c2-80bd-87afde15e03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:49.328110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721deec3-834c-43c2-80bd-87afde15e03b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:49.328399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721deec3-834c-43c2-80bd-87afde15e03b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:49.328681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721deec3-834c-43c2-80bd-87afde15e03b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:49.338522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721deec3-834c-43c2-80bd-87afde15e03b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-721deec3-834c-43c2-80bd-87afde15e03b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:47:49.346617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:49.347742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:49.348802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 573/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:47:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:49.353786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb59f08-1a5f-4869-b89a-9b7e2495d635 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:49.405705 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:49.451678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb59f08-1a5f-4869-b89a-9b7e2495d635 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:49.484196 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58284122-49ba-40ae-af21-6c680d23abf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:49.484852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58284122-49ba-40ae-af21-6c680d23abf2 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:49.485132 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58284122-49ba-40ae-af21-6c680d23abf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:49.485403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58284122-49ba-40ae-af21-6c680d23abf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:49.486141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58284122-49ba-40ae-af21-6c680d23abf2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:49.486471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58284122-49ba-40ae-af21-6c680d23abf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:49.487832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58284122-49ba-40ae-af21-6c680d23abf2 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:49.495579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 577/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:47:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:49.501438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721deec3-834c-43c2-80bd-87afde15e03b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:49.598595 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721deec3-834c-43c2-80bd-87afde15e03b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:49.620474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:49.631350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca71948-cdbc-4f49-a079-8a7a6caa49f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:49.632224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca71948-cdbc-4f49-a079-8a7a6caa49f0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:49.632507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca71948-cdbc-4f49-a079-8a7a6caa49f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:49.632787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca71948-cdbc-4f49-a079-8a7a6caa49f0 None admin] PATH_INFO: `/v3/users/d8ce622e5c3c454b830534c208bae585` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:49.634227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca71948-cdbc-4f49-a079-8a7a6caa49f0 None admin] RBAC: Authorizing `identity:get_user(user_id=d8ce622e5c3c454b830534c208bae585)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:49.635681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca71948-cdbc-4f49-a079-8a7a6caa49f0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:49.637827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 574/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:49 2022] GET /identity/v3/users/d8ce622e5c3c454b830534c208bae585 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:49.643490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58284122-49ba-40ae-af21-6c680d23abf2 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:49.739857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58284122-49ba-40ae-af21-6c680d23abf2 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:49.753190 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:49.773873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e16f41-f65c-44d4-829a-fc472c8666a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:49.774748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e16f41-f65c-44d4-829a-fc472c8666a9 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:49.775029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e16f41-f65c-44d4-829a-fc472c8666a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:49.775302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e16f41-f65c-44d4-829a-fc472c8666a9 None admin] PATH_INFO: `/v3/projects/071cbe4e324e4c45b4142e58e6bb3b14` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:49.776450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e16f41-f65c-44d4-829a-fc472c8666a9 None admin] RBAC: Authorizing `identity:get_project(project_id=071cbe4e324e4c45b4142e58e6bb3b14)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:49.777903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e16f41-f65c-44d4-829a-fc472c8666a9 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:49.779595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 578/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:47:49 2022] GET /identity/v3/projects/071cbe4e324e4c45b4142e58e6bb3b14 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:49.785636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca71948-cdbc-4f49-a079-8a7a6caa49f0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:49.881069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca71948-cdbc-4f49-a079-8a7a6caa49f0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:49.912766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e366eb-0231-4466-82a1-857d4928dfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:49.913642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e366eb-0231-4466-82a1-857d4928dfb5 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:49.913923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e366eb-0231-4466-82a1-857d4928dfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:49.914195 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e366eb-0231-4466-82a1-857d4928dfb5 None admin] PATH_INFO: `/v3/projects/071cbe4e324e4c45b4142e58e6bb3b14/users/d8ce622e5c3c454b830534c208bae585/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:49.916136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e366eb-0231-4466-82a1-857d4928dfb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=071cbe4e324e4c45b4142e58e6bb3b14, user_id=d8ce622e5c3c454b830534c208bae585, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:49.917537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e366eb-0231-4466-82a1-857d4928dfb5 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:49.930841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 575/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:47:49 2022] PUT /identity/v3/projects/071cbe4e324e4c45b4142e58e6bb3b14/users/d8ce622e5c3c454b830534c208bae585/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:47:52.063531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d6132-9f3f-403b-81f8-aa18c2f37795 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:52.064061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d6132-9f3f-403b-81f8-aa18c2f37795 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:52.064061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d6132-9f3f-403b-81f8-aa18c2f37795 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:52.065433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 579/1154] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:52.070709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4340d4-f8ab-425d-af21-2b8682965430 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:52.071075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4340d4-f8ab-425d-af21-2b8682965430 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:52.071269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4340d4-f8ab-425d-af21-2b8682965430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:52.088207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-eb4340d4-f8ab-425d-af21-2b8682965430 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:52.089386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-eb4340d4-f8ab-425d-af21-2b8682965430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:52.133917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:52.153872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 576/1155] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:52.177158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d6132-9f3f-403b-81f8-aa18c2f37795 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:52.276807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d6132-9f3f-403b-81f8-aa18c2f37795 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:52.310745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242c871-772d-4bd1-bff7-bc4921b98e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:52.311531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242c871-772d-4bd1-bff7-bc4921b98e6c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:52.311817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242c871-772d-4bd1-bff7-bc4921b98e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:52.312093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242c871-772d-4bd1-bff7-bc4921b98e6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:52.322148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242c871-772d-4bd1-bff7-bc4921b98e6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-4242c871-772d-4bd1-bff7-bc4921b98e6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:47:52.330260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:52.331369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:47:52.332602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 580/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:47:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:47:52.337503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4340d4-f8ab-425d-af21-2b8682965430 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:52.433380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4340d4-f8ab-425d-af21-2b8682965430 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:52.467036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7c84f-a1f6-46de-b3b9-331773e15f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:52.467699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7c84f-a1f6-46de-b3b9-331773e15f12 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:52.467989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7c84f-a1f6-46de-b3b9-331773e15f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:52.468267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7c84f-a1f6-46de-b3b9-331773e15f12 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:52.469009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7c84f-a1f6-46de-b3b9-331773e15f12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:52.469316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7c84f-a1f6-46de-b3b9-331773e15f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:52.470720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7c84f-a1f6-46de-b3b9-331773e15f12 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:52.478534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 577/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:47:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:52.484695 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242c871-772d-4bd1-bff7-bc4921b98e6c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:52.526223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:52.580743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242c871-772d-4bd1-bff7-bc4921b98e6c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:52.612560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90172a0d-e380-4f97-9e3a-0396dfdf0c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:52.613410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90172a0d-e380-4f97-9e3a-0396dfdf0c2c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:52.613686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90172a0d-e380-4f97-9e3a-0396dfdf0c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:52.613953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90172a0d-e380-4f97-9e3a-0396dfdf0c2c None admin] PATH_INFO: `/v3/users/d8ce622e5c3c454b830534c208bae585` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:52.615151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90172a0d-e380-4f97-9e3a-0396dfdf0c2c None admin] RBAC: Authorizing `identity:get_user(user_id=d8ce622e5c3c454b830534c208bae585)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:52.616580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90172a0d-e380-4f97-9e3a-0396dfdf0c2c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:52.618172 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 581/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:47:52 2022] GET /identity/v3/users/d8ce622e5c3c454b830534c208bae585 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:52.624469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7c84f-a1f6-46de-b3b9-331773e15f12 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:52.670317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:52.723248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7c84f-a1f6-46de-b3b9-331773e15f12 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:52.755427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672e952-9fba-47e8-b319-e1ed18a29abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:52.756073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e952-9fba-47e8-b319-e1ed18a29abc None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:52.756361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e952-9fba-47e8-b319-e1ed18a29abc None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:52.756640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e952-9fba-47e8-b319-e1ed18a29abc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:52.820937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672e952-9fba-47e8-b319-e1ed18a29abc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:52.822406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672e952-9fba-47e8-b319-e1ed18a29abc None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:52.865617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:52.921774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 578/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:47:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:47:52.928745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90172a0d-e380-4f97-9e3a-0396dfdf0c2c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:53.024798 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90172a0d-e380-4f97-9e3a-0396dfdf0c2c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:53.035139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:53.058616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382648bc-6dcc-4ecc-b060-59fbe5c8f241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:53.059461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382648bc-6dcc-4ecc-b060-59fbe5c8f241 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:53.059746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382648bc-6dcc-4ecc-b060-59fbe5c8f241 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:53.060023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382648bc-6dcc-4ecc-b060-59fbe5c8f241 None admin] PATH_INFO: `/v3/projects/071cbe4e324e4c45b4142e58e6bb3b14` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:53.061193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382648bc-6dcc-4ecc-b060-59fbe5c8f241 None admin] RBAC: Authorizing `identity:get_project(project_id=071cbe4e324e4c45b4142e58e6bb3b14)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:53.062665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382648bc-6dcc-4ecc-b060-59fbe5c8f241 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:53.064333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 582/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:47:52 2022] GET /identity/v3/projects/071cbe4e324e4c45b4142e58e6bb3b14 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:53.070335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672e952-9fba-47e8-b319-e1ed18a29abc None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:53.165876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672e952-9fba-47e8-b319-e1ed18a29abc None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:53.198163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf1ce1-8204-432d-88b1-54aef2be1701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:53.198911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1ce1-8204-432d-88b1-54aef2be1701 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:53.199717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1ce1-8204-432d-88b1-54aef2be1701 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:53.199995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1ce1-8204-432d-88b1-54aef2be1701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:53.200819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf1ce1-8204-432d-88b1-54aef2be1701 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=071cbe4e324e4c45b4142e58e6bb3b14, user.id=d8ce622e5c3c454b830534c208bae585) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:53.201134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf1ce1-8204-432d-88b1-54aef2be1701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:53.202583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf1ce1-8204-432d-88b1-54aef2be1701 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:53.208224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 579/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:47:53 2022] GET /identity/v3/role_assignments?user.id=d8ce622e5c3c454b830534c208bae585&scope.project.id=071cbe4e324e4c45b4142e58e6bb3b14&role.id=0c473678b281495f86f58b077be04e1b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:55.493565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f7968-d1c2-4751-9c14-8527374538aa None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:55.494126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f7968-d1c2-4751-9c14-8527374538aa None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:55.494126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f7968-d1c2-4751-9c14-8527374538aa None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:55.495514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 583/1162] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:55.500711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c20758-8ec7-4699-aa6b-34b5f4a9261e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:55.501153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c20758-8ec7-4699-aa6b-34b5f4a9261e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:55.501279 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c20758-8ec7-4699-aa6b-34b5f4a9261e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:55.518422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-85c20758-8ec7-4699-aa6b-34b5f4a9261e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:55.519612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-85c20758-8ec7-4699-aa6b-34b5f4a9261e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:55.521523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:55.584777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 580/1163] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:55.607026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f7968-d1c2-4751-9c14-8527374538aa None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:55.704210 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f7968-d1c2-4751-9c14-8527374538aa None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:55.736757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a36d5-34e5-4a8c-b31e-cb4e13d4baf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:55.737393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a36d5-34e5-4a8c-b31e-cb4e13d4baf6 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:55.737674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a36d5-34e5-4a8c-b31e-cb4e13d4baf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:55.737953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a36d5-34e5-4a8c-b31e-cb4e13d4baf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:55.782021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:55.804041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a36d5-34e5-4a8c-b31e-cb4e13d4baf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:55.805489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a36d5-34e5-4a8c-b31e-cb4e13d4baf6 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:55.899279 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 584/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:47:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:47:55.905666 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c20758-8ec7-4699-aa6b-34b5f4a9261e None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:56.000199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c20758-8ec7-4699-aa6b-34b5f4a9261e None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:56.032738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2afa35-9315-46c3-81bb-de485b51cdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:56.033545 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2afa35-9315-46c3-81bb-de485b51cdcb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:56.033826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2afa35-9315-46c3-81bb-de485b51cdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:56.034099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2afa35-9315-46c3-81bb-de485b51cdcb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:56.040802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2afa35-9315-46c3-81bb-de485b51cdcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:56.042243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2afa35-9315-46c3-81bb-de485b51cdcb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:56.044513 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-6d2afa35-9315-46c3-81bb-de485b51cdcb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:47:56.058589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:47:56.059662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:47:56.060769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 581/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 12:47:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:47:56.065729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a36d5-34e5-4a8c-b31e-cb4e13d4baf6 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:56.140027 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:47:56.160905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a36d5-34e5-4a8c-b31e-cb4e13d4baf6 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:56.193090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcf653-9723-4131-b258-ab26c430ae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:56.193729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcf653-9723-4131-b258-ab26c430ae90 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:56.194006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcf653-9723-4131-b258-ab26c430ae90 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:56.194275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcf653-9723-4131-b258-ab26c430ae90 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:56.195003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcf653-9723-4131-b258-ab26c430ae90 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:47:56.195302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcf653-9723-4131-b258-ab26c430ae90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:56.196650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcf653-9723-4131-b258-ab26c430ae90 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:56.203981 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 585/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 12:47:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:47:58.341482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630b6a-b98f-4fb8-991d-cb0a275fd1c9 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:58.342009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630b6a-b98f-4fb8-991d-cb0a275fd1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:58.342009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630b6a-b98f-4fb8-991d-cb0a275fd1c9 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:58.343458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 582/1167] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:47:58.348502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab396ff8-d908-4345-8cd1-2b68520ba34f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:58.348934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab396ff8-d908-4345-8cd1-2b68520ba34f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:58.349036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab396ff8-d908-4345-8cd1-2b68520ba34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:58.365733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ab396ff8-d908-4345-8cd1-2b68520ba34f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:47:58.366924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ab396ff8-d908-4345-8cd1-2b68520ba34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:47:58.431089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 586/1168] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:47:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:47:58.453562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40630b6a-b98f-4fb8-991d-cb0a275fd1c9 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:47:58.550300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40630b6a-b98f-4fb8-991d-cb0a275fd1c9 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:47:58.582506 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306dfddb-949e-4a24-a8b8-1ca975f6b747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:47:58.583151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dfddb-949e-4a24-a8b8-1ca975f6b747 None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:47:58.583433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dfddb-949e-4a24-a8b8-1ca975f6b747 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:47:58.583710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dfddb-949e-4a24-a8b8-1ca975f6b747 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:47:58.584447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306dfddb-949e-4a24-a8b8-1ca975f6b747 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:47:58.585820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306dfddb-949e-4a24-a8b8-1ca975f6b747 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:47:58.600371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 583/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Tue May 24 12:47:58 2022] POST /identity/v3/domains => generated 267 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:48:00.827052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547158a-0557-4365-a08c-31ee32be8193 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:00.827584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547158a-0557-4365-a08c-31ee32be8193 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:00.827584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547158a-0557-4365-a08c-31ee32be8193 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:00.828972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 587/1170] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:48:00.834234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acd910-3326-47e3-97c3-261f6a3f773f None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:00.834636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acd910-3326-47e3-97c3-261f6a3f773f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:00.834837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acd910-3326-47e3-97c3-261f6a3f773f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:00.851401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-09acd910-3326-47e3-97c3-261f6a3f773f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:48:00.852575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-09acd910-3326-47e3-97c3-261f6a3f773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:48:00.902839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:00.916544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 584/1171] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:48:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:48:00.941016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547158a-0557-4365-a08c-31ee32be8193 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:01.010230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:01.039755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547158a-0557-4365-a08c-31ee32be8193 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:01.072283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e17981-81b4-4923-b387-3d485f8611ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:01.073139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e17981-81b4-4923-b387-3d485f8611ca None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:01.073426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e17981-81b4-4923-b387-3d485f8611ca None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:01.073730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e17981-81b4-4923-b387-3d485f8611ca None admin] PATH_INFO: `/v3/domains/378b3d0fb7694470a3fd0a95c6003b83` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:01.084133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e17981-81b4-4923-b387-3d485f8611ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=378b3d0fb7694470a3fd0a95c6003b83)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:01.085533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e17981-81b4-4923-b387-3d485f8611ca None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:01.087270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 588/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Tue May 24 12:48:00 2022] GET /identity/v3/domains/378b3d0fb7694470a3fd0a95c6003b83 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:01.094629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acd910-3326-47e3-97c3-261f6a3f773f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:01.190200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acd910-3326-47e3-97c3-261f6a3f773f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:01.222818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2e850-0ed5-48da-849f-3ab64a82a7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:01.223479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2e850-0ed5-48da-849f-3ab64a82a7ca None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:01.223768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2e850-0ed5-48da-849f-3ab64a82a7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:01.224048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2e850-0ed5-48da-849f-3ab64a82a7ca None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:01.224750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2e850-0ed5-48da-849f-3ab64a82a7ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:01.226178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2e850-0ed5-48da-849f-3ab64a82a7ca None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:01.251051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 585/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:48:01 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:48:03.435188 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d883d-c597-421e-b3ce-79022249549e None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:03.435597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d883d-c597-421e-b3ce-79022249549e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:03.435742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d883d-c597-421e-b3ce-79022249549e None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:03.437125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 589/1174] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:48:03.442400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cf3fc-ef27-42a5-add3-1c700dff0a67 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:03.442827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cf3fc-ef27-42a5-add3-1c700dff0a67 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:03.442966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cf3fc-ef27-42a5-add3-1c700dff0a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:03.460021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-358cf3fc-ef27-42a5-add3-1c700dff0a67 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:48:03.461200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-358cf3fc-ef27-42a5-add3-1c700dff0a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:48:03.495284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:03.531169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 586/1175] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:48:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:48:03.556417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d883d-c597-421e-b3ce-79022249549e None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:03.654657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d883d-c597-421e-b3ce-79022249549e None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:03.682342 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:03.688114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888c032-d5d7-4214-be7b-789ccc307da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:03.688968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888c032-d5d7-4214-be7b-789ccc307da5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:03.689250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888c032-d5d7-4214-be7b-789ccc307da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:03.689520 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888c032-d5d7-4214-be7b-789ccc307da5 None admin] PATH_INFO: `/v3/domains/378b3d0fb7694470a3fd0a95c6003b83` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:03.690708 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888c032-d5d7-4214-be7b-789ccc307da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=378b3d0fb7694470a3fd0a95c6003b83)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:03.692126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888c032-d5d7-4214-be7b-789ccc307da5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:03.693728 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 590/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Tue May 24 12:48:03 2022] GET /identity/v3/domains/378b3d0fb7694470a3fd0a95c6003b83 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:03.699412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cf3fc-ef27-42a5-add3-1c700dff0a67 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:03.802758 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cf3fc-ef27-42a5-add3-1c700dff0a67 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:03.836775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4abbe-94ce-43dd-8925-eb7b70b4113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:03.837443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4abbe-94ce-43dd-8925-eb7b70b4113f None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:03.837731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4abbe-94ce-43dd-8925-eb7b70b4113f None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:03.838004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4abbe-94ce-43dd-8925-eb7b70b4113f None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:03.838733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4abbe-94ce-43dd-8925-eb7b70b4113f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:03.840249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4abbe-94ce-43dd-8925-eb7b70b4113f None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:03.860250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 587/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:48:03 2022] POST /identity/v3/users => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:48:06.084274 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db8c02-9893-44a6-aee4-28c90856be4a None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:06.084832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db8c02-9893-44a6-aee4-28c90856be4a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:06.084832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db8c02-9893-44a6-aee4-28c90856be4a None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:06.086200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 591/1178] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:48:06.091471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f5c74-1525-475b-85e6-ac58e240d317 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:06.091884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f5c74-1525-475b-85e6-ac58e240d317 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:06.092067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f5c74-1525-475b-85e6-ac58e240d317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:06.108894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-366f5c74-1525-475b-85e6-ac58e240d317 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:48:06.110052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-366f5c74-1525-475b-85e6-ac58e240d317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:48:06.129687 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:06.175388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 588/1179] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:48:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:48:06.198063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db8c02-9893-44a6-aee4-28c90856be4a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:06.298134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db8c02-9893-44a6-aee4-28c90856be4a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:06.330221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c469ddf-5791-44a7-a27e-d864f7edc735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:06.331022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469ddf-5791-44a7-a27e-d864f7edc735 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:06.331309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469ddf-5791-44a7-a27e-d864f7edc735 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:06.331588 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469ddf-5791-44a7-a27e-d864f7edc735 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:06.341942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c469ddf-5791-44a7-a27e-d864f7edc735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-3c469ddf-5791-44a7-a27e-d864f7edc735 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:48:06.349493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:48:06.350613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:48:06.351679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 592/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:48:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:48:06.357137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f5c74-1525-475b-85e6-ac58e240d317 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:06.451145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f5c74-1525-475b-85e6-ac58e240d317 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:06.483464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32bc9f-2619-4f1d-b694-43b5c8c7ebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:06.484123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32bc9f-2619-4f1d-b694-43b5c8c7ebaf None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:06.484409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32bc9f-2619-4f1d-b694-43b5c8c7ebaf None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:06.484688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32bc9f-2619-4f1d-b694-43b5c8c7ebaf None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:06.485433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32bc9f-2619-4f1d-b694-43b5c8c7ebaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:48:06.485739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32bc9f-2619-4f1d-b694-43b5c8c7ebaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:06.487134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32bc9f-2619-4f1d-b694-43b5c8c7ebaf None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:06.494841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 589/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:48:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:06.501255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c469ddf-5791-44a7-a27e-d864f7edc735 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:06.559106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:06.599381 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c469ddf-5791-44a7-a27e-d864f7edc735 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:06.632409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ad22d-0aff-40d6-be3c-98a920d7de3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:06.633262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ad22d-0aff-40d6-be3c-98a920d7de3e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:06.633807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ad22d-0aff-40d6-be3c-98a920d7de3e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:06.634086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ad22d-0aff-40d6-be3c-98a920d7de3e None admin] PATH_INFO: `/v3/users/3a882f2f195240dbadee6d8005a2582c` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:06.647544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ad22d-0aff-40d6-be3c-98a920d7de3e None admin] RBAC: Authorizing `identity:get_user(user_id=3a882f2f195240dbadee6d8005a2582c)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:06.648990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ad22d-0aff-40d6-be3c-98a920d7de3e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:06.650645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 593/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:48:06 2022] GET /identity/v3/users/3a882f2f195240dbadee6d8005a2582c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:06.656179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32bc9f-2619-4f1d-b694-43b5c8c7ebaf None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:06.675961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:06.758917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32bc9f-2619-4f1d-b694-43b5c8c7ebaf None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:06.790385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822d415-6dd2-4fef-b361-44133d7743fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:06.791024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822d415-6dd2-4fef-b361-44133d7743fe None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:06.791307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822d415-6dd2-4fef-b361-44133d7743fe None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:06.791582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822d415-6dd2-4fef-b361-44133d7743fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:06.856020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822d415-6dd2-4fef-b361-44133d7743fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:06.857382 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822d415-6dd2-4fef-b361-44133d7743fe None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:06.875213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:06.952877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 590/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:48:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:48:06.958734 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ad22d-0aff-40d6-be3c-98a920d7de3e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:07.054673 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ad22d-0aff-40d6-be3c-98a920d7de3e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:07.071054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:07.087316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2becfc9-f2ee-4250-9edb-f4f6fb996a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:07.088167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2becfc9-f2ee-4250-9edb-f4f6fb996a1a None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:07.088451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2becfc9-f2ee-4250-9edb-f4f6fb996a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:07.088728 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2becfc9-f2ee-4250-9edb-f4f6fb996a1a None admin] PATH_INFO: `/v3/projects/9bc408f9be01467790ca9308e79e35c1` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:07.090464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2becfc9-f2ee-4250-9edb-f4f6fb996a1a None admin] RBAC: Authorizing `identity:get_project(project_id=9bc408f9be01467790ca9308e79e35c1)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:07.091898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2becfc9-f2ee-4250-9edb-f4f6fb996a1a None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:07.093555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 594/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:48:06 2022] GET /identity/v3/projects/9bc408f9be01467790ca9308e79e35c1 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:07.099398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822d415-6dd2-4fef-b361-44133d7743fe None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:07.194349 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822d415-6dd2-4fef-b361-44133d7743fe None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:07.210886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:07.227639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06497fa-c44e-488b-b3fb-e9cd5c3c00b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:07.228285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06497fa-c44e-488b-b3fb-e9cd5c3c00b8 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:07.228568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06497fa-c44e-488b-b3fb-e9cd5c3c00b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:07.228842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06497fa-c44e-488b-b3fb-e9cd5c3c00b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:07.229659 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06497fa-c44e-488b-b3fb-e9cd5c3c00b8 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=9bc408f9be01467790ca9308e79e35c1, user.id=3a882f2f195240dbadee6d8005a2582c) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:48:07.230543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06497fa-c44e-488b-b3fb-e9cd5c3c00b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:07.231958 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06497fa-c44e-488b-b3fb-e9cd5c3c00b8 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:07.237259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 591/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:48:07 2022] GET /identity/v3/role_assignments?user.id=3a882f2f195240dbadee6d8005a2582c&scope.project.id=9bc408f9be01467790ca9308e79e35c1&role.id=0c473678b281495f86f58b077be04e1b => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:09.389946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c77b29-e396-4461-b34e-1510ad6d737c None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:09.390494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c77b29-e396-4461-b34e-1510ad6d737c None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:09.390610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c77b29-e396-4461-b34e-1510ad6d737c None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:09.391919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 595/1186] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:48:09.397130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4e3ef-3c8f-4840-ade4-6381dc6f2cbd None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:09.397520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4e3ef-3c8f-4840-ade4-6381dc6f2cbd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:09.397692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4e3ef-3c8f-4840-ade4-6381dc6f2cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:09.414267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7cb4e3ef-3c8f-4840-ade4-6381dc6f2cbd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:48:09.415472 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7cb4e3ef-3c8f-4840-ade4-6381dc6f2cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:48:09.478731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 592/1187] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:48:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:48:09.504725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c77b29-e396-4461-b34e-1510ad6d737c None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:09.602219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c77b29-e396-4461-b34e-1510ad6d737c None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:09.635066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82ba53-5b2e-40fa-8247-6d3602f76c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:09.636351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82ba53-5b2e-40fa-8247-6d3602f76c99 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:09.636666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82ba53-5b2e-40fa-8247-6d3602f76c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:09.636939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82ba53-5b2e-40fa-8247-6d3602f76c99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:09.646672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82ba53-5b2e-40fa-8247-6d3602f76c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-9c82ba53-5b2e-40fa-8247-6d3602f76c99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:48:09.654665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:48:09.655773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:48:09.656700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 596/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:48:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:48:09.661532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4e3ef-3c8f-4840-ade4-6381dc6f2cbd None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:09.756611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4e3ef-3c8f-4840-ade4-6381dc6f2cbd None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:09.771622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:09.792491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c8c0a-73e9-4a3c-821f-abfdbfcfd39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:09.793171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c8c0a-73e9-4a3c-821f-abfdbfcfd39c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:09.793461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c8c0a-73e9-4a3c-821f-abfdbfcfd39c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:09.793739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c8c0a-73e9-4a3c-821f-abfdbfcfd39c None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:09.794514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c8c0a-73e9-4a3c-821f-abfdbfcfd39c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:48:09.794825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c8c0a-73e9-4a3c-821f-abfdbfcfd39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:09.796197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c8c0a-73e9-4a3c-821f-abfdbfcfd39c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:09.805010 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 593/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:48:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:09.811075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82ba53-5b2e-40fa-8247-6d3602f76c99 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:09.863135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:09.917942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82ba53-5b2e-40fa-8247-6d3602f76c99 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:09.952635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b71971-2eda-49ea-be16-b4725e5f0b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:09.953507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b71971-2eda-49ea-be16-b4725e5f0b09 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:09.953788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b71971-2eda-49ea-be16-b4725e5f0b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:09.954059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b71971-2eda-49ea-be16-b4725e5f0b09 None admin] PATH_INFO: `/v3/users/3a882f2f195240dbadee6d8005a2582c` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:09.955258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b71971-2eda-49ea-be16-b4725e5f0b09 None admin] RBAC: Authorizing `identity:get_user(user_id=3a882f2f195240dbadee6d8005a2582c)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:09.956692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b71971-2eda-49ea-be16-b4725e5f0b09 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:09.958335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 597/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:48:09 2022] GET /identity/v3/users/3a882f2f195240dbadee6d8005a2582c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:09.964029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c8c0a-73e9-4a3c-821f-abfdbfcfd39c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:10.060517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c8c0a-73e9-4a3c-821f-abfdbfcfd39c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:10.092490 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b46a6-54b6-4994-ac8e-86bc17092a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:10.093337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b46a6-54b6-4994-ac8e-86bc17092a38 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:10.093618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b46a6-54b6-4994-ac8e-86bc17092a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:10.093890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b46a6-54b6-4994-ac8e-86bc17092a38 None admin] PATH_INFO: `/v3/projects/9bc408f9be01467790ca9308e79e35c1` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:10.095072 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b46a6-54b6-4994-ac8e-86bc17092a38 None admin] RBAC: Authorizing `identity:get_project(project_id=9bc408f9be01467790ca9308e79e35c1)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:10.096494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b46a6-54b6-4994-ac8e-86bc17092a38 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:10.098153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 594/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:48:09 2022] GET /identity/v3/projects/9bc408f9be01467790ca9308e79e35c1 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:10.104045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b71971-2eda-49ea-be16-b4725e5f0b09 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:10.201536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b71971-2eda-49ea-be16-b4725e5f0b09 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:10.214611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:10.234153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab88cf-476b-4253-9ee0-d54de52d1069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:10.235058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab88cf-476b-4253-9ee0-d54de52d1069 None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:10.235344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab88cf-476b-4253-9ee0-d54de52d1069 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:10.235623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab88cf-476b-4253-9ee0-d54de52d1069 None admin] PATH_INFO: `/v3/projects/9bc408f9be01467790ca9308e79e35c1/users/3a882f2f195240dbadee6d8005a2582c/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:10.237575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab88cf-476b-4253-9ee0-d54de52d1069 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bc408f9be01467790ca9308e79e35c1, user_id=3a882f2f195240dbadee6d8005a2582c, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:10.239049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab88cf-476b-4253-9ee0-d54de52d1069 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:10.251221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 598/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:48:10 2022] PUT /identity/v3/projects/9bc408f9be01467790ca9308e79e35c1/users/3a882f2f195240dbadee6d8005a2582c/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:48:12.405446 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e52a7-50b6-4f8b-a0f3-ed519426fa26 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:12.405909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e52a7-50b6-4f8b-a0f3-ed519426fa26 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:12.406011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e52a7-50b6-4f8b-a0f3-ed519426fa26 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:12.407400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 595/1193] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:48:12.412615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694dfec-59ab-44d4-9542-344c311b5af9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:12.413020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694dfec-59ab-44d4-9542-344c311b5af9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:12.413178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694dfec-59ab-44d4-9542-344c311b5af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:12.429644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9694dfec-59ab-44d4-9542-344c311b5af9 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:48:12.430848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9694dfec-59ab-44d4-9542-344c311b5af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:48:12.494109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 599/1194] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:48:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:48:12.518054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e52a7-50b6-4f8b-a0f3-ed519426fa26 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:12.560731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:12.615902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e52a7-50b6-4f8b-a0f3-ed519426fa26 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:12.648360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574342d8-20c2-456f-907d-a94a49f9a8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:12.649138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574342d8-20c2-456f-907d-a94a49f9a8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:12.649423 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574342d8-20c2-456f-907d-a94a49f9a8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:12.649700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574342d8-20c2-456f-907d-a94a49f9a8e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:12.660069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574342d8-20c2-456f-907d-a94a49f9a8e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-574342d8-20c2-456f-907d-a94a49f9a8e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:48:12.668090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:48:12.669038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:48:12.670023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 596/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:48:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:48:12.675083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694dfec-59ab-44d4-9542-344c311b5af9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:12.769713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694dfec-59ab-44d4-9542-344c311b5af9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:12.771977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:12.804158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76460900-358f-4972-965b-f5b5b92d609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:12.804808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76460900-358f-4972-965b-f5b5b92d609d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:12.805093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76460900-358f-4972-965b-f5b5b92d609d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:12.805365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76460900-358f-4972-965b-f5b5b92d609d None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:12.806099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76460900-358f-4972-965b-f5b5b92d609d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:48:12.806435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76460900-358f-4972-965b-f5b5b92d609d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:12.808376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76460900-358f-4972-965b-f5b5b92d609d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:12.816085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 600/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:48:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:12.821641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574342d8-20c2-456f-907d-a94a49f9a8e2 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:12.915292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574342d8-20c2-456f-907d-a94a49f9a8e2 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:12.916746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:12.948984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7e695-dfcb-4712-9b85-391a665bdf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:12.949846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7e695-dfcb-4712-9b85-391a665bdf4c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:12.950127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7e695-dfcb-4712-9b85-391a665bdf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:12.950436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7e695-dfcb-4712-9b85-391a665bdf4c None admin] PATH_INFO: `/v3/users/3a882f2f195240dbadee6d8005a2582c` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:12.951592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee7e695-dfcb-4712-9b85-391a665bdf4c None admin] RBAC: Authorizing `identity:get_user(user_id=3a882f2f195240dbadee6d8005a2582c)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:12.953021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee7e695-dfcb-4712-9b85-391a665bdf4c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:12.954650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 597/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 12:48:12 2022] GET /identity/v3/users/3a882f2f195240dbadee6d8005a2582c => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:12.960484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76460900-358f-4972-965b-f5b5b92d609d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:13.057530 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76460900-358f-4972-965b-f5b5b92d609d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:13.089355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2784d3-a9b3-429b-8ac8-da15933448cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:13.089998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2784d3-a9b3-429b-8ac8-da15933448cf None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:13.090311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2784d3-a9b3-429b-8ac8-da15933448cf None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:13.090592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2784d3-a9b3-429b-8ac8-da15933448cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:13.122779 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:13.156416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2784d3-a9b3-429b-8ac8-da15933448cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:13.157860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2784d3-a9b3-429b-8ac8-da15933448cf None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:13.251890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 601/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:48:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:48:13.257784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7e695-dfcb-4712-9b85-391a665bdf4c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:13.353361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7e695-dfcb-4712-9b85-391a665bdf4c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:13.385594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c7be4-2603-4aa1-ab5d-1cef86a21817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:13.387031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c7be4-2603-4aa1-ab5d-1cef86a21817 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:13.387317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c7be4-2603-4aa1-ab5d-1cef86a21817 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:13.387596 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c7be4-2603-4aa1-ab5d-1cef86a21817 None admin] PATH_INFO: `/v3/projects/9bc408f9be01467790ca9308e79e35c1` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:13.388776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c7be4-2603-4aa1-ab5d-1cef86a21817 None admin] RBAC: Authorizing `identity:get_project(project_id=9bc408f9be01467790ca9308e79e35c1)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:13.390202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c7be4-2603-4aa1-ab5d-1cef86a21817 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:13.391899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 598/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 12:48:13 2022] GET /identity/v3/projects/9bc408f9be01467790ca9308e79e35c1 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:13.397787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2784d3-a9b3-429b-8ac8-da15933448cf None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:48:13.460875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:13.494559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2784d3-a9b3-429b-8ac8-da15933448cf None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:48:13.528036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98919150-5344-4eb2-ba8d-533e179e1a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:48:13.528627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98919150-5344-4eb2-ba8d-533e179e1a6e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:48:13.528914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98919150-5344-4eb2-ba8d-533e179e1a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:48:13.529192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98919150-5344-4eb2-ba8d-533e179e1a6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:48:13.530011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98919150-5344-4eb2-ba8d-533e179e1a6e None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=9bc408f9be01467790ca9308e79e35c1, user.id=3a882f2f195240dbadee6d8005a2582c) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:48:13.530352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98919150-5344-4eb2-ba8d-533e179e1a6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:48:13.531785 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98919150-5344-4eb2-ba8d-533e179e1a6e None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:48:13.537584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 602/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:48:13 2022] GET /identity/v3/role_assignments?user.id=3a882f2f195240dbadee6d8005a2582c&scope.project.id=9bc408f9be01467790ca9308e79e35c1&role.id=0c473678b281495f86f58b077be04e1b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:48:23.381291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=81 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:48:23.535474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=35 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:13.832544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcaf90-09e3-4dbc-b505-bc4128463610 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:13.832544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcaf90-09e3-4dbc-b505-bc4128463610 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:13.832544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcaf90-09e3-4dbc-b505-bc4128463610 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:13.836703 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 599/1201] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:49:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:49:13.842998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4815488-ae03-4a2e-ba39-9eb9938f3f02 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:13.843510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4815488-ae03-4a2e-ba39-9eb9938f3f02 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:13.843608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4815488-ae03-4a2e-ba39-9eb9938f3f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:13.860948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-d4815488-ae03-4a2e-ba39-9eb9938f3f02 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:49:13.862134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d4815488-ae03-4a2e-ba39-9eb9938f3f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:49:13.951697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 603/1202] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:49:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:49:13.979673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcaf90-09e3-4dbc-b505-bc4128463610 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:14.126644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcaf90-09e3-4dbc-b505-bc4128463610 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:14.173692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe36842-816d-4668-ba83-a53a9edfd966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:14.174850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36842-816d-4668-ba83-a53a9edfd966 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:14.175292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36842-816d-4668-ba83-a53a9edfd966 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:14.175913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36842-816d-4668-ba83-a53a9edfd966 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:14.194635 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe36842-816d-4668-ba83-a53a9edfd966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:14.196206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe36842-816d-4668-ba83-a53a9edfd966 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-fbe36842-816d-4668-ba83-a53a9edfd966 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:14.219350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:14.220716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:14.221888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:14.221888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:14.221888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:14.221888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:14.221888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 600/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:14.227651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4815488-ae03-4a2e-ba39-9eb9938f3f02 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:14.366229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4815488-ae03-4a2e-ba39-9eb9938f3f02 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:14.414920 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:14.423325 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451efe0-65cc-4b58-9ee2-ebbdccc36e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:14.424575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451efe0-65cc-4b58-9ee2-ebbdccc36e91 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:14.424575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451efe0-65cc-4b58-9ee2-ebbdccc36e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:14.424773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451efe0-65cc-4b58-9ee2-ebbdccc36e91 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:14.425505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451efe0-65cc-4b58-9ee2-ebbdccc36e91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:14.425825 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451efe0-65cc-4b58-9ee2-ebbdccc36e91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:14.427610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451efe0-65cc-4b58-9ee2-ebbdccc36e91 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:14.449533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 604/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:14.451067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe36842-816d-4668-ba83-a53a9edfd966 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:14.528435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:14.588262 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe36842-816d-4668-ba83-a53a9edfd966 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:14.637178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e60dcb-d399-4257-a824-abdde4118fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:14.638059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e60dcb-d399-4257-a824-abdde4118fac None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:14.638406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e60dcb-d399-4257-a824-abdde4118fac None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:14.638730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e60dcb-d399-4257-a824-abdde4118fac None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:14.639499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e60dcb-d399-4257-a824-abdde4118fac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:14.641072 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e60dcb-d399-4257-a824-abdde4118fac None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:14.666408 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 601/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 12:49:14 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:49:17.363510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad055eb-f076-4d93-a032-8af01adebfd5 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:17.365029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad055eb-f076-4d93-a032-8af01adebfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:17.365029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad055eb-f076-4d93-a032-8af01adebfd5 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:17.365467 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 605/1206] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:49:17.372333 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a5a76-3640-4867-b0dc-8d001a08186b None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:17.372752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a5a76-3640-4867-b0dc-8d001a08186b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:17.372868 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a5a76-3640-4867-b0dc-8d001a08186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:17.393919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-098a5a76-3640-4867-b0dc-8d001a08186b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:49:17.395236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-098a5a76-3640-4867-b0dc-8d001a08186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:49:17.471874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 602/1207] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:49:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:49:17.497542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad055eb-f076-4d93-a032-8af01adebfd5 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:17.612365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad055eb-f076-4d93-a032-8af01adebfd5 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:17.650136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f231969-96c4-4918-baaf-645de4c6839c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:17.650924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f231969-96c4-4918-baaf-645de4c6839c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:17.651357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f231969-96c4-4918-baaf-645de4c6839c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:17.651624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f231969-96c4-4918-baaf-645de4c6839c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:17.663437 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f231969-96c4-4918-baaf-645de4c6839c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-5f231969-96c4-4918-baaf-645de4c6839c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:49:17.672640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:17.673841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:49:17.675163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:17.678545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 606/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:49:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:17.682721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a5a76-3640-4867-b0dc-8d001a08186b None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:17.722984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:17.803825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a5a76-3640-4867-b0dc-8d001a08186b None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:17.845008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01efc51-bcc7-4704-a84a-238b35a33dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:17.845717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01efc51-bcc7-4704-a84a-238b35a33dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:17.846031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01efc51-bcc7-4704-a84a-238b35a33dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:17.846351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01efc51-bcc7-4704-a84a-238b35a33dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:17.847220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01efc51-bcc7-4704-a84a-238b35a33dc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:17.847581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01efc51-bcc7-4704-a84a-238b35a33dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:17.849109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01efc51-bcc7-4704-a84a-238b35a33dc7 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:17.864086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 603/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:49:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:17.865295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f231969-96c4-4918-baaf-645de4c6839c None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:17.921334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:17.982139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f231969-96c4-4918-baaf-645de4c6839c None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:18.025248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dcc23-a23d-4070-91bb-6b4f3428cf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:18.026473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dcc23-a23d-4070-91bb-6b4f3428cf07 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:18.026473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dcc23-a23d-4070-91bb-6b4f3428cf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:18.026652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dcc23-a23d-4070-91bb-6b4f3428cf07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:18.037538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dcc23-a23d-4070-91bb-6b4f3428cf07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:18.038997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dcc23-a23d-4070-91bb-6b4f3428cf07 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-8a5dcc23-a23d-4070-91bb-6b4f3428cf07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:18.057486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:18.058611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:18.059659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:18.059659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:18.059659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:18.059659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:18.059659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 607/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:18.064851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01efc51-bcc7-4704-a84a-238b35a33dc7 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:18.181331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:18.184343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01efc51-bcc7-4704-a84a-238b35a33dc7 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:18.237563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76302d7-5e73-4265-b7e5-8df52da2e762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:18.239006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76302d7-5e73-4265-b7e5-8df52da2e762 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:18.239366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76302d7-5e73-4265-b7e5-8df52da2e762 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:18.241895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76302d7-5e73-4265-b7e5-8df52da2e762 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:18.241895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76302d7-5e73-4265-b7e5-8df52da2e762 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:18.241895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76302d7-5e73-4265-b7e5-8df52da2e762 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:18.243741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76302d7-5e73-4265-b7e5-8df52da2e762 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:18.263428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 604/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:49:18.269496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dcc23-a23d-4070-91bb-6b4f3428cf07 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:18.393794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dcc23-a23d-4070-91bb-6b4f3428cf07 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:18.401989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:18.444476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08014f-64a0-4139-a29c-71f5a280b7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:18.446965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08014f-64a0-4139-a29c-71f5a280b7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:18.446965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08014f-64a0-4139-a29c-71f5a280b7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:18.446965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08014f-64a0-4139-a29c-71f5a280b7a1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:18.454491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08014f-64a0-4139-a29c-71f5a280b7a1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:18.455936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08014f-64a0-4139-a29c-71f5a280b7a1 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-3c08014f-64a0-4139-a29c-71f5a280b7a1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:49:18.471720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:18.472733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:18.480116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:18.480116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:18.480116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:49:18.480116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:49:18.480116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:49:18.480116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:49:18.480116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:49:18.480116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:49:18.480116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:18.480116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:18.480116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 608/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:18.480745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76302d7-5e73-4265-b7e5-8df52da2e762 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:18.601135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76302d7-5e73-4265-b7e5-8df52da2e762 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:18.648872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda32d6-966e-46ea-9625-d0b479bc4ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:18.649843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda32d6-966e-46ea-9625-d0b479bc4ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:18.650299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda32d6-966e-46ea-9625-d0b479bc4ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:18.650591 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda32d6-966e-46ea-9625-d0b479bc4ed8 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:18.651580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda32d6-966e-46ea-9625-d0b479bc4ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:18.651891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda32d6-966e-46ea-9625-d0b479bc4ed8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:18.654919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda32d6-966e-46ea-9625-d0b479bc4ed8 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:18.683997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 605/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 12:49:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:18.690875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08014f-64a0-4139-a29c-71f5a280b7a1 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:18.809238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08014f-64a0-4139-a29c-71f5a280b7a1 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:18.845574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6fbfe-d132-4e28-a47c-bdb0a1417882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:18.846230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6fbfe-d132-4e28-a47c-bdb0a1417882 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:18.846539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6fbfe-d132-4e28-a47c-bdb0a1417882 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:18.846817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6fbfe-d132-4e28-a47c-bdb0a1417882 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:18.881195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:18.924959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6fbfe-d132-4e28-a47c-bdb0a1417882 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:18.926460 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6fbfe-d132-4e28-a47c-bdb0a1417882 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:19.033407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 609/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:49:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:49:19.040086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda32d6-966e-46ea-9625-d0b479bc4ed8 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:19.059452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:19.149318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda32d6-966e-46ea-9625-d0b479bc4ed8 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:19.194370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1941b-2bdb-4e4c-91c3-500e7e5b9f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:19.195398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1941b-2bdb-4e4c-91c3-500e7e5b9f81 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:19.195923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1941b-2bdb-4e4c-91c3-500e7e5b9f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:19.196310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1941b-2bdb-4e4c-91c3-500e7e5b9f81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:19.213857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1941b-2bdb-4e4c-91c3-500e7e5b9f81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:19.215813 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1941b-2bdb-4e4c-91c3-500e7e5b9f81 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-87a1941b-2bdb-4e4c-91c3-500e7e5b9f81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:19.237556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:19.239090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:19.240023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:19.240023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:19.240023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:19.240023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:19.240023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 606/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:49:19.250115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6fbfe-d132-4e28-a47c-bdb0a1417882 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:19.318461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:19.365727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6fbfe-d132-4e28-a47c-bdb0a1417882 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:19.408291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0aee96-4672-40fc-9918-dce69803e9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:19.409004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0aee96-4672-40fc-9918-dce69803e9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:19.409347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0aee96-4672-40fc-9918-dce69803e9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:19.409623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0aee96-4672-40fc-9918-dce69803e9d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:19.410383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0aee96-4672-40fc-9918-dce69803e9d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:19.410690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0aee96-4672-40fc-9918-dce69803e9d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:19.412193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0aee96-4672-40fc-9918-dce69803e9d2 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:19.426691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 610/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:19.432811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1941b-2bdb-4e4c-91c3-500e7e5b9f81 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:19.517574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:19.554995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1941b-2bdb-4e4c-91c3-500e7e5b9f81 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:19.591757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11975e-0014-4bc9-9149-fed75972419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:19.592624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11975e-0014-4bc9-9149-fed75972419b None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:19.592977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11975e-0014-4bc9-9149-fed75972419b None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:19.593334 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11975e-0014-4bc9-9149-fed75972419b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:19.600758 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11975e-0014-4bc9-9149-fed75972419b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:19.602297 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11975e-0014-4bc9-9149-fed75972419b None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-ac11975e-0014-4bc9-9149-fed75972419b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:49:19.617892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:19.619322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:19.620462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:49:19.620462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:49:19.620462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:49:19.620462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:49:19.620462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:49:19.620462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:49:19.620462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:19.620462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:19.620462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 607/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:49:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:19.627628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0aee96-4672-40fc-9918-dce69803e9d2 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:19.746718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0aee96-4672-40fc-9918-dce69803e9d2 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:19.777302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:19.789266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10636e-664d-4aff-8a7f-0d042834af55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:19.790042 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10636e-664d-4aff-8a7f-0d042834af55 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:19.790348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10636e-664d-4aff-8a7f-0d042834af55 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:19.790692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10636e-664d-4aff-8a7f-0d042834af55 None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:19.791552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10636e-664d-4aff-8a7f-0d042834af55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:19.791915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10636e-664d-4aff-8a7f-0d042834af55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:19.793446 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10636e-664d-4aff-8a7f-0d042834af55 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:19.811371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 611/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:49:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:19.820378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11975e-0014-4bc9-9149-fed75972419b None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:19.941092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11975e-0014-4bc9-9149-fed75972419b None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:19.973114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:19.981319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbf18d-5f06-4aaf-b81c-824ae13d26ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:19.982326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf18d-5f06-4aaf-b81c-824ae13d26ad None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:19.983847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf18d-5f06-4aaf-b81c-824ae13d26ad None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:19.983847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf18d-5f06-4aaf-b81c-824ae13d26ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:19.983847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbf18d-5f06-4aaf-b81c-824ae13d26ad None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=630f0527bd844de9be072c459dab96c9) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:19.984269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbf18d-5f06-4aaf-b81c-824ae13d26ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:19.986164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbf18d-5f06-4aaf-b81c-824ae13d26ad None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:19.995100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 608/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:49:19 2022] GET /identity/v3/role_assignments?user.id=630f0527bd844de9be072c459dab96c9&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:22.361730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19797b45-2691-482e-8396-5016ef2f6d88 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:22.362327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19797b45-2691-482e-8396-5016ef2f6d88 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:22.362327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19797b45-2691-482e-8396-5016ef2f6d88 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:22.363684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 612/1220] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:49:22.369550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9e4e1-c08f-4b07-b206-c909fb1e9dcd None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:22.370066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9e4e1-c08f-4b07-b206-c909fb1e9dcd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:22.370178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9e4e1-c08f-4b07-b206-c909fb1e9dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:22.388617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-95a9e4e1-c08f-4b07-b206-c909fb1e9dcd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:49:22.389798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-95a9e4e1-c08f-4b07-b206-c909fb1e9dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:49:22.461154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 609/1221] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:49:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:49:22.484251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19797b45-2691-482e-8396-5016ef2f6d88 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:22.601293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19797b45-2691-482e-8396-5016ef2f6d88 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:22.641149 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274de27-86fa-4f32-a2fd-0c3e7854751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:22.641883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274de27-86fa-4f32-a2fd-0c3e7854751e None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:22.642162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274de27-86fa-4f32-a2fd-0c3e7854751e None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:22.642524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274de27-86fa-4f32-a2fd-0c3e7854751e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:22.652856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274de27-86fa-4f32-a2fd-0c3e7854751e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-6274de27-86fa-4f32-a2fd-0c3e7854751e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:49:22.660872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:22.662017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:22.662924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 613/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:49:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:22.669159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9e4e1-c08f-4b07-b206-c909fb1e9dcd None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:22.777528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9e4e1-c08f-4b07-b206-c909fb1e9dcd None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:22.806951 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:22.819464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5edfec-727d-445d-80fb-dd4b743b4506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:22.820156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5edfec-727d-445d-80fb-dd4b743b4506 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:22.820434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5edfec-727d-445d-80fb-dd4b743b4506 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:22.820709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5edfec-727d-445d-80fb-dd4b743b4506 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:22.821456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5edfec-727d-445d-80fb-dd4b743b4506 None admin] RBAC: Adding query filter params (name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:22.821760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5edfec-727d-445d-80fb-dd4b743b4506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:22.823145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5edfec-727d-445d-80fb-dd4b743b4506 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:22.831076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 610/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:49:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:22.838162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274de27-86fa-4f32-a2fd-0c3e7854751e None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:22.891145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:22.952983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274de27-86fa-4f32-a2fd-0c3e7854751e None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:22.992528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4019676-8785-469a-83a8-075d2c6dcaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:22.993560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4019676-8785-469a-83a8-075d2c6dcaec None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:22.993884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4019676-8785-469a-83a8-075d2c6dcaec None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:22.994259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4019676-8785-469a-83a8-075d2c6dcaec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:23.007302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4019676-8785-469a-83a8-075d2c6dcaec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:23.009320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4019676-8785-469a-83a8-075d2c6dcaec None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-a4019676-8785-469a-83a8-075d2c6dcaec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:23.026113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:23.027151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:23.028057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:23.028057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:23.028057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:23.028057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:23.028057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 614/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:23.033079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5edfec-727d-445d-80fb-dd4b743b4506 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:23.148695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5edfec-727d-445d-80fb-dd4b743b4506 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:23.198617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5075d6d-5250-4e51-b7f9-b946e70f22cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:23.199538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5075d6d-5250-4e51-b7f9-b946e70f22cf None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:23.200014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5075d6d-5250-4e51-b7f9-b946e70f22cf None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:23.200196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5075d6d-5250-4e51-b7f9-b946e70f22cf None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:23.200934 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5075d6d-5250-4e51-b7f9-b946e70f22cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:23.201239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5075d6d-5250-4e51-b7f9-b946e70f22cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:23.202646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5075d6d-5250-4e51-b7f9-b946e70f22cf None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:23.225479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 611/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:23.227486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4019676-8785-469a-83a8-075d2c6dcaec None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:23.331035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:23.339902 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4019676-8785-469a-83a8-075d2c6dcaec None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:23.374822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc5b2-6dbc-4140-95a5-e3899f5638b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:23.375636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc5b2-6dbc-4140-95a5-e3899f5638b5 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:23.375984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc5b2-6dbc-4140-95a5-e3899f5638b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:23.376330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc5b2-6dbc-4140-95a5-e3899f5638b5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:23.385459 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cc5b2-6dbc-4140-95a5-e3899f5638b5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:23.387136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cc5b2-6dbc-4140-95a5-e3899f5638b5 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-130cc5b2-6dbc-4140-95a5-e3899f5638b5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:49:23.402409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:23.403591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:23.404759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:23.404759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:23.404759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:49:23.404759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:49:23.404759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:49:23.404759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:49:23.404759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:49:23.404759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:49:23.404759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:23.404759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:23.404759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 615/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:49:23.410038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5075d6d-5250-4e51-b7f9-b946e70f22cf None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:23.459797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:23.516995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5075d6d-5250-4e51-b7f9-b946e70f22cf None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:23.563009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92135066-3f97-446a-bd4a-093861fd864f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:23.563714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92135066-3f97-446a-bd4a-093861fd864f None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:23.563995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92135066-3f97-446a-bd4a-093861fd864f None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:23.564270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92135066-3f97-446a-bd4a-093861fd864f None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:23.565106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92135066-3f97-446a-bd4a-093861fd864f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:23.565405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92135066-3f97-446a-bd4a-093861fd864f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:23.566913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92135066-3f97-446a-bd4a-093861fd864f None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:23.590646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 612/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 12:49:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:23.596478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc5b2-6dbc-4140-95a5-e3899f5638b5 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:23.706474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc5b2-6dbc-4140-95a5-e3899f5638b5 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:23.743945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab15d95-4c20-4ad0-afd4-07932993dcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:23.744928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab15d95-4c20-4ad0-afd4-07932993dcf2 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:23.745251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab15d95-4c20-4ad0-afd4-07932993dcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:23.745591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab15d95-4c20-4ad0-afd4-07932993dcf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:23.756965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab15d95-4c20-4ad0-afd4-07932993dcf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:23.758926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab15d95-4c20-4ad0-afd4-07932993dcf2 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-fab15d95-4c20-4ad0-afd4-07932993dcf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:23.781533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:23.782663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:23.783592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:23.783592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:23.783592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:23.783592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:23.783592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 616/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:23.790001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92135066-3f97-446a-bd4a-093861fd864f None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:23.894712 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92135066-3f97-446a-bd4a-093861fd864f None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:23.897062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:23.935199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f995918-8807-4d79-a7a0-a1f4728ad21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:23.936155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f995918-8807-4d79-a7a0-a1f4728ad21d None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:23.936499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f995918-8807-4d79-a7a0-a1f4728ad21d None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:23.936780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f995918-8807-4d79-a7a0-a1f4728ad21d None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:23.937544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f995918-8807-4d79-a7a0-a1f4728ad21d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:23.937879 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f995918-8807-4d79-a7a0-a1f4728ad21d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:23.939575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f995918-8807-4d79-a7a0-a1f4728ad21d None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:23.952617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 613/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:23.958258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab15d95-4c20-4ad0-afd4-07932993dcf2 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:23.967833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:24.068169 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab15d95-4c20-4ad0-afd4-07932993dcf2 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:24.104066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ac3d1-a0ee-4926-9b95-1fc5df6d7d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:24.104987 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac3d1-a0ee-4926-9b95-1fc5df6d7d8d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:24.105345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac3d1-a0ee-4926-9b95-1fc5df6d7d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:24.105704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac3d1-a0ee-4926-9b95-1fc5df6d7d8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:24.113557 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ac3d1-a0ee-4926-9b95-1fc5df6d7d8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:24.115043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ac3d1-a0ee-4926-9b95-1fc5df6d7d8d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-908ac3d1-a0ee-4926-9b95-1fc5df6d7d8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:49:24.130200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:24.131587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:24.139646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:49:24.139646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:49:24.139646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:49:24.139646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:49:24.139646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:49:24.139646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:49:24.139646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:24.139646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:24.139646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 617/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:49:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:24.140251 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f995918-8807-4d79-a7a0-a1f4728ad21d None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:24.267452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f995918-8807-4d79-a7a0-a1f4728ad21d None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:24.308104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af590e43-5803-42ac-8164-e12055d07ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:24.308763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af590e43-5803-42ac-8164-e12055d07ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:24.309060 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af590e43-5803-42ac-8164-e12055d07ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:24.309333 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af590e43-5803-42ac-8164-e12055d07ad3 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:24.310101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af590e43-5803-42ac-8164-e12055d07ad3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:24.310490 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af590e43-5803-42ac-8164-e12055d07ad3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:24.311981 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af590e43-5803-42ac-8164-e12055d07ad3 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:24.324695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 614/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:49:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:24.330755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ac3d1-a0ee-4926-9b95-1fc5df6d7d8d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:24.402670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:24.439760 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ac3d1-a0ee-4926-9b95-1fc5df6d7d8d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:24.474788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad04df-a5ca-4426-9ea8-e52afb6d883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:24.475781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad04df-a5ca-4426-9ea8-e52afb6d883f None admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:24.476128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad04df-a5ca-4426-9ea8-e52afb6d883f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:24.476416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad04df-a5ca-4426-9ea8-e52afb6d883f None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/630f0527bd844de9be072c459dab96c9/roles/1ea1fd1a01f6487aac4236749de7d4e1` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:24.491753 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad04df-a5ca-4426-9ea8-e52afb6d883f None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=630f0527bd844de9be072c459dab96c9, role_id=1ea1fd1a01f6487aac4236749de7d4e1)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:24.493715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad04df-a5ca-4426-9ea8-e52afb6d883f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:24.507281 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 618/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:49:24 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/630f0527bd844de9be072c459dab96c9/roles/1ea1fd1a01f6487aac4236749de7d4e1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:49:26.834403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08078-5dcb-4a0a-a5b5-3d983bea3199 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:26.835002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08078-5dcb-4a0a-a5b5-3d983bea3199 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:26.835002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08078-5dcb-4a0a-a5b5-3d983bea3199 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:26.837271 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 615/1233] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:49:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:49:26.843164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922bcdc-b947-4f1d-80f8-88650829ba3e None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:26.843636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922bcdc-b947-4f1d-80f8-88650829ba3e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:26.843739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922bcdc-b947-4f1d-80f8-88650829ba3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:26.862005 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b922bcdc-b947-4f1d-80f8-88650829ba3e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:49:26.863668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b922bcdc-b947-4f1d-80f8-88650829ba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:49:26.940548 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 619/1234] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:49:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:49:26.965899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a08078-5dcb-4a0a-a5b5-3d983bea3199 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:26.992515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:27.079003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a08078-5dcb-4a0a-a5b5-3d983bea3199 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:27.116250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d9b19-ac1f-4e8e-8949-cd8743157f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:27.116998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d9b19-ac1f-4e8e-8949-cd8743157f08 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:27.117280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d9b19-ac1f-4e8e-8949-cd8743157f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:27.117586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d9b19-ac1f-4e8e-8949-cd8743157f08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:27.127902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d9b19-ac1f-4e8e-8949-cd8743157f08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-d74d9b19-ac1f-4e8e-8949-cd8743157f08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:49:27.137570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:27.138920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:27.140054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 616/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 12:49:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:27.144820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922bcdc-b947-4f1d-80f8-88650829ba3e None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:27.199098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:27.259196 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922bcdc-b947-4f1d-80f8-88650829ba3e None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:27.295858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f21849-de2b-4318-a966-502b597248de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:27.296593 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f21849-de2b-4318-a966-502b597248de None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:27.296880 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f21849-de2b-4318-a966-502b597248de None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:27.297201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f21849-de2b-4318-a966-502b597248de None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:27.297998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f21849-de2b-4318-a966-502b597248de None admin] RBAC: Adding query filter params (name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:27.298334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f21849-de2b-4318-a966-502b597248de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:27.299722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f21849-de2b-4318-a966-502b597248de None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:27.309042 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 620/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 12:49:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:27.315251 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d9b19-ac1f-4e8e-8949-cd8743157f08 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:27.408569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:27.425388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d9b19-ac1f-4e8e-8949-cd8743157f08 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:27.461714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ed256-1132-4d95-b5d9-e324ec446000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:27.462525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ed256-1132-4d95-b5d9-e324ec446000 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:27.462809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ed256-1132-4d95-b5d9-e324ec446000 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:27.463095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ed256-1132-4d95-b5d9-e324ec446000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:27.474670 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ed256-1132-4d95-b5d9-e324ec446000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:27.476194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ed256-1132-4d95-b5d9-e324ec446000 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-2e1ed256-1132-4d95-b5d9-e324ec446000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:27.496036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:27.497157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:27.498110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:27.498110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:27.498110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:27.498110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:27.498110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 617/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:49:27.502915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f21849-de2b-4318-a966-502b597248de None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:27.615876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f21849-de2b-4318-a966-502b597248de None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:27.629819 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:27.667632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d93643-3ca0-45eb-9d73-5b35e3640912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:27.668627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d93643-3ca0-45eb-9d73-5b35e3640912 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:27.669170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d93643-3ca0-45eb-9d73-5b35e3640912 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:27.669725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d93643-3ca0-45eb-9d73-5b35e3640912 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:27.670651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d93643-3ca0-45eb-9d73-5b35e3640912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:27.671096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d93643-3ca0-45eb-9d73-5b35e3640912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:27.673054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d93643-3ca0-45eb-9d73-5b35e3640912 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:27.686085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 621/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:49:27.692108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ed256-1132-4d95-b5d9-e324ec446000 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:27.799280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ed256-1132-4d95-b5d9-e324ec446000 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:27.835136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c53e67-bc2e-4c73-980b-07e5a0665225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:27.836140 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c53e67-bc2e-4c73-980b-07e5a0665225 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:27.836526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c53e67-bc2e-4c73-980b-07e5a0665225 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:27.836917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c53e67-bc2e-4c73-980b-07e5a0665225 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:27.840154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:27.847114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c53e67-bc2e-4c73-980b-07e5a0665225 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:27.848835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c53e67-bc2e-4c73-980b-07e5a0665225 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-32c53e67-bc2e-4c73-980b-07e5a0665225 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:49:27.865847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:27.867110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:27.868226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:27.868226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:27.868226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:49:27.868226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:49:27.868226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:49:27.868226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:49:27.868226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:49:27.868226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:49:27.868226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:27.868226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:27.868226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 618/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:27.874171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d93643-3ca0-45eb-9d73-5b35e3640912 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:27.986816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d93643-3ca0-45eb-9d73-5b35e3640912 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:28.024505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0205a-841d-4502-8c19-f554e594849f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:28.025308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0205a-841d-4502-8c19-f554e594849f None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:28.025618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0205a-841d-4502-8c19-f554e594849f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:28.025989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0205a-841d-4502-8c19-f554e594849f None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:28.026955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0205a-841d-4502-8c19-f554e594849f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:28.027334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0205a-841d-4502-8c19-f554e594849f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:28.029013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0205a-841d-4502-8c19-f554e594849f None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:28.053306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 622/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 12:49:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:28.055059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c53e67-bc2e-4c73-980b-07e5a0665225 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:28.167122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c53e67-bc2e-4c73-980b-07e5a0665225 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:28.205815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d2227-2f7a-4a14-8a9c-57ab0a51aabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:28.206507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d2227-2f7a-4a14-8a9c-57ab0a51aabb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:28.206796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d2227-2f7a-4a14-8a9c-57ab0a51aabb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:28.207073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d2227-2f7a-4a14-8a9c-57ab0a51aabb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:28.283052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:28.291037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d2227-2f7a-4a14-8a9c-57ab0a51aabb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:28.292585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d2227-2f7a-4a14-8a9c-57ab0a51aabb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:28.411083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 619/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:49:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:49:28.418507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0205a-841d-4502-8c19-f554e594849f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:28.451689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:28.537150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0205a-841d-4502-8c19-f554e594849f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:28.577456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b95e7d-1bcf-4506-ba38-d4c6162fd615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:28.578434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b95e7d-1bcf-4506-ba38-d4c6162fd615 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:28.578721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b95e7d-1bcf-4506-ba38-d4c6162fd615 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:28.579025 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b95e7d-1bcf-4506-ba38-d4c6162fd615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:28.590787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b95e7d-1bcf-4506-ba38-d4c6162fd615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:28.592503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b95e7d-1bcf-4506-ba38-d4c6162fd615 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-c8b95e7d-1bcf-4506-ba38-d4c6162fd615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:28.612541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:28.613699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:28.620135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:28.620135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:28.620135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:28.620135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:28.620135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 623/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:28.620942 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d2227-2f7a-4a14-8a9c-57ab0a51aabb None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:28.723205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:28.730991 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d2227-2f7a-4a14-8a9c-57ab0a51aabb None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:28.766421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302d8b0-89e2-4287-8a17-ba1dfef292cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:28.767091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302d8b0-89e2-4287-8a17-ba1dfef292cd None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:28.767385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302d8b0-89e2-4287-8a17-ba1dfef292cd None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:28.767645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302d8b0-89e2-4287-8a17-ba1dfef292cd None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:28.768368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302d8b0-89e2-4287-8a17-ba1dfef292cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:28.768670 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302d8b0-89e2-4287-8a17-ba1dfef292cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:28.770045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302d8b0-89e2-4287-8a17-ba1dfef292cd None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:28.785046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 620/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:28.791600 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b95e7d-1bcf-4506-ba38-d4c6162fd615 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:28.880026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:28.903154 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b95e7d-1bcf-4506-ba38-d4c6162fd615 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:28.938737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e5d97-5af1-43c2-a715-2154fb4559ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:28.939727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e5d97-5af1-43c2-a715-2154fb4559ff None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:28.940030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e5d97-5af1-43c2-a715-2154fb4559ff None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:28.940401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e5d97-5af1-43c2-a715-2154fb4559ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:28.948430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e5d97-5af1-43c2-a715-2154fb4559ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:28.949909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e5d97-5af1-43c2-a715-2154fb4559ff None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-fe9e5d97-5af1-43c2-a715-2154fb4559ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:49:28.967645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:28.968817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:28.969947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:49:28.969947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:49:28.969947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:49:28.969947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:49:28.969947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:49:28.969947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:49:28.969947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:28.969947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:28.969947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 624/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:49:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 12:49:28.975832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302d8b0-89e2-4287-8a17-ba1dfef292cd None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:29.088905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302d8b0-89e2-4287-8a17-ba1dfef292cd None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:29.124447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690646f-eb24-4a79-926e-d562a4c7104e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:29.125109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690646f-eb24-4a79-926e-d562a4c7104e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:29.125392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690646f-eb24-4a79-926e-d562a4c7104e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:29.125663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690646f-eb24-4a79-926e-d562a4c7104e None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:29.126674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690646f-eb24-4a79-926e-d562a4c7104e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:29.127137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690646f-eb24-4a79-926e-d562a4c7104e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:29.128987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690646f-eb24-4a79-926e-d562a4c7104e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:29.148031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 621/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:49:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:29.150160 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e5d97-5af1-43c2-a715-2154fb4559ff None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:29.284521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e5d97-5af1-43c2-a715-2154fb4559ff None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:29.324427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28d6f5-d89e-4aac-8a01-f7d7d1afbb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:29.325091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28d6f5-d89e-4aac-8a01-f7d7d1afbb14 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:29.325375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28d6f5-d89e-4aac-8a01-f7d7d1afbb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:29.325648 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28d6f5-d89e-4aac-8a01-f7d7d1afbb14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:29.326514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28d6f5-d89e-4aac-8a01-f7d7d1afbb14 None admin] RBAC: Adding query filter params (role.id=1ea1fd1a01f6487aac4236749de7d4e1, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=630f0527bd844de9be072c459dab96c9) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:29.326828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28d6f5-d89e-4aac-8a01-f7d7d1afbb14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:29.328262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28d6f5-d89e-4aac-8a01-f7d7d1afbb14 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:29.335027 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 625/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:49:29 2022] GET /identity/v3/role_assignments?user.id=630f0527bd844de9be072c459dab96c9&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=1ea1fd1a01f6487aac4236749de7d4e1 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:31.945705 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a190bee-4de7-461c-a148-5efdfb50baa1 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:31.946235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a190bee-4de7-461c-a148-5efdfb50baa1 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:31.946235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a190bee-4de7-461c-a148-5efdfb50baa1 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:31.947649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 622/1247] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:49:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:49:31.953544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d31c4a-b615-4a79-9610-dbe2e97c50af None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:31.954028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d31c4a-b615-4a79-9610-dbe2e97c50af None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:31.954122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d31c4a-b615-4a79-9610-dbe2e97c50af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:31.959124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:31.980526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c7d31c4a-b615-4a79-9610-dbe2e97c50af None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:49:31.982118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c7d31c4a-b615-4a79-9610-dbe2e97c50af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:49:32.059147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 626/1248] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:49:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:49:32.082237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a190bee-4de7-461c-a148-5efdfb50baa1 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:32.100286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:32.197854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a190bee-4de7-461c-a148-5efdfb50baa1 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:32.237498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2574d0-09e5-4e0e-977d-430f0e81bbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:32.238383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2574d0-09e5-4e0e-977d-430f0e81bbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:32.238609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2574d0-09e5-4e0e-977d-430f0e81bbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:32.238883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2574d0-09e5-4e0e-977d-430f0e81bbf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:32.251563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2574d0-09e5-4e0e-977d-430f0e81bbf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-3d2574d0-09e5-4e0e-977d-430f0e81bbf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:49:32.262403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:32.263403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:32.264540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 623/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:49:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:32.269847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d31c4a-b615-4a79-9610-dbe2e97c50af None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:32.397203 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d31c4a-b615-4a79-9610-dbe2e97c50af None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:32.428971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:32.435498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2cc77-3f43-42d4-a317-51e031b6bda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:32.436505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2cc77-3f43-42d4-a317-51e031b6bda4 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:32.436809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2cc77-3f43-42d4-a317-51e031b6bda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:32.437093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2cc77-3f43-42d4-a317-51e031b6bda4 None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:32.438004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2cc77-3f43-42d4-a317-51e031b6bda4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:32.438332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2cc77-3f43-42d4-a317-51e031b6bda4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:32.439726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2cc77-3f43-42d4-a317-51e031b6bda4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:32.448921 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 627/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:49:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:32.454782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2574d0-09e5-4e0e-977d-430f0e81bbf1 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:32.539985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:32.565864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2574d0-09e5-4e0e-977d-430f0e81bbf1 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:32.608877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfec415-4899-4a5d-a06a-577dec6f1cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:32.610474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfec415-4899-4a5d-a06a-577dec6f1cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:32.611139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfec415-4899-4a5d-a06a-577dec6f1cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:32.611696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfec415-4899-4a5d-a06a-577dec6f1cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:32.636035 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfec415-4899-4a5d-a06a-577dec6f1cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:32.637536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfec415-4899-4a5d-a06a-577dec6f1cd7 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-bbfec415-4899-4a5d-a06a-577dec6f1cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:32.656618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:32.657628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:32.658641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:32.658641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:32.658641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:32.658641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:32.658641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 624/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:32.663857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2cc77-3f43-42d4-a317-51e031b6bda4 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:32.769405 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2cc77-3f43-42d4-a317-51e031b6bda4 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:32.803900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a14302-2433-43ec-a6f7-db80f78ae2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:32.805099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a14302-2433-43ec-a6f7-db80f78ae2db None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:32.805381 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a14302-2433-43ec-a6f7-db80f78ae2db None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:32.805654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a14302-2433-43ec-a6f7-db80f78ae2db None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:32.806422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a14302-2433-43ec-a6f7-db80f78ae2db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:32.806731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a14302-2433-43ec-a6f7-db80f78ae2db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:32.808135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a14302-2433-43ec-a6f7-db80f78ae2db None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:32.822510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 628/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:32.828184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfec415-4899-4a5d-a06a-577dec6f1cd7 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:32.933091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfec415-4899-4a5d-a06a-577dec6f1cd7 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:32.960248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:32.969005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ccd9b-01b6-4219-9a4e-bbcdfff42fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:32.969829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ccd9b-01b6-4219-9a4e-bbcdfff42fcf None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:32.970111 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ccd9b-01b6-4219-9a4e-bbcdfff42fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:32.970417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ccd9b-01b6-4219-9a4e-bbcdfff42fcf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:32.979583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ccd9b-01b6-4219-9a4e-bbcdfff42fcf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:32.981074 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ccd9b-01b6-4219-9a4e-bbcdfff42fcf None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-2e0ccd9b-01b6-4219-9a4e-bbcdfff42fcf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:49:32.997314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:32.998534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:32.999590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:32.999590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:32.999590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:49:32.999590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:49:32.999590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:49:32.999590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:49:32.999590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:49:32.999590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:49:32.999590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:32.999590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:32.999590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 625/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:33.004470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a14302-2433-43ec-a6f7-db80f78ae2db None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:33.062612 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:33.111334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a14302-2433-43ec-a6f7-db80f78ae2db None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:33.145479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734d7fe-9fc8-44ef-b1fe-0541d59f4f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:33.146155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734d7fe-9fc8-44ef-b1fe-0541d59f4f5d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:33.146474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734d7fe-9fc8-44ef-b1fe-0541d59f4f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:33.146756 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734d7fe-9fc8-44ef-b1fe-0541d59f4f5d None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:33.147597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734d7fe-9fc8-44ef-b1fe-0541d59f4f5d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:33.148464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734d7fe-9fc8-44ef-b1fe-0541d59f4f5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:33.149913 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734d7fe-9fc8-44ef-b1fe-0541d59f4f5d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:33.167711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 629/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 12:49:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:33.173826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ccd9b-01b6-4219-9a4e-bbcdfff42fcf None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:33.286233 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ccd9b-01b6-4219-9a4e-bbcdfff42fcf None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:33.320629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc194ce-5d3d-48e2-8544-ba05bc96ad76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:33.321216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc194ce-5d3d-48e2-8544-ba05bc96ad76 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:33.321504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc194ce-5d3d-48e2-8544-ba05bc96ad76 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:33.321780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc194ce-5d3d-48e2-8544-ba05bc96ad76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:33.374350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:33.387312 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc194ce-5d3d-48e2-8544-ba05bc96ad76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:33.388752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc194ce-5d3d-48e2-8544-ba05bc96ad76 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:33.482624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 626/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:49:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:49:33.488310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734d7fe-9fc8-44ef-b1fe-0541d59f4f5d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:33.583029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734d7fe-9fc8-44ef-b1fe-0541d59f4f5d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:33.595281 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:33.615897 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d6b67-bb44-4226-8469-d7abf2e5a642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:33.617195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d6b67-bb44-4226-8469-d7abf2e5a642 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:33.617484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d6b67-bb44-4226-8469-d7abf2e5a642 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:33.617754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d6b67-bb44-4226-8469-d7abf2e5a642 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:33.627278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d6b67-bb44-4226-8469-d7abf2e5a642 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:33.628705 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d6b67-bb44-4226-8469-d7abf2e5a642 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-200d6b67-bb44-4226-8469-d7abf2e5a642 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:33.645611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:33.646682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:33.647860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:33.647860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:33.647860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:33.647860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:33.647860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 630/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:33.652405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc194ce-5d3d-48e2-8544-ba05bc96ad76 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:33.735459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:33.747104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc194ce-5d3d-48e2-8544-ba05bc96ad76 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:33.780224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be880cfb-2ef9-4552-bbeb-f79a3901270c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:33.780871 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be880cfb-2ef9-4552-bbeb-f79a3901270c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:33.781156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be880cfb-2ef9-4552-bbeb-f79a3901270c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:33.781432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be880cfb-2ef9-4552-bbeb-f79a3901270c None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:33.782148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be880cfb-2ef9-4552-bbeb-f79a3901270c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:33.782476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be880cfb-2ef9-4552-bbeb-f79a3901270c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:33.783829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be880cfb-2ef9-4552-bbeb-f79a3901270c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:33.794522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 627/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:33.799891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d6b67-bb44-4226-8469-d7abf2e5a642 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:33.894073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d6b67-bb44-4226-8469-d7abf2e5a642 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:33.926006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daa465-7c19-4d1b-a120-9dbe2ad7dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:33.926799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa465-7c19-4d1b-a120-9dbe2ad7dd65 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:33.927080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa465-7c19-4d1b-a120-9dbe2ad7dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:33.927355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa465-7c19-4d1b-a120-9dbe2ad7dd65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:33.933683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20daa465-7c19-4d1b-a120-9dbe2ad7dd65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:33.935130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20daa465-7c19-4d1b-a120-9dbe2ad7dd65 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-20daa465-7c19-4d1b-a120-9dbe2ad7dd65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:49:33.948345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:33.949414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:33.950611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:49:33.950611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:49:33.950611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:49:33.950611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:49:33.950611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:49:33.950611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:49:33.950611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:33.950611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:33.950611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 631/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:49:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:33.956822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be880cfb-2ef9-4552-bbeb-f79a3901270c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:34.053872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be880cfb-2ef9-4552-bbeb-f79a3901270c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:34.086043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5fe7f-0000-4554-929f-7ce228084c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:34.086720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5fe7f-0000-4554-929f-7ce228084c14 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:34.087006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5fe7f-0000-4554-929f-7ce228084c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:34.087279 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5fe7f-0000-4554-929f-7ce228084c14 None admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:34.088051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5fe7f-0000-4554-929f-7ce228084c14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:34.088360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5fe7f-0000-4554-929f-7ce228084c14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:34.089771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5fe7f-0000-4554-929f-7ce228084c14 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:34.101818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 628/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:49:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:49:34.107791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daa465-7c19-4d1b-a120-9dbe2ad7dd65 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:34.132999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:34.212623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daa465-7c19-4d1b-a120-9dbe2ad7dd65 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:34.254087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb3cdc-034f-4da2-97c7-d86aa7119277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:34.254779 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb3cdc-034f-4da2-97c7-d86aa7119277 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:34.255066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb3cdc-034f-4da2-97c7-d86aa7119277 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:34.255342 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb3cdc-034f-4da2-97c7-d86aa7119277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:34.256445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb3cdc-034f-4da2-97c7-d86aa7119277 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=630f0527bd844de9be072c459dab96c9) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:34.256758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb3cdc-034f-4da2-97c7-d86aa7119277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:34.258391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb3cdc-034f-4da2-97c7-d86aa7119277 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:34.263792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 632/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:49:34 2022] GET /identity/v3/role_assignments?user.id=630f0527bd844de9be072c459dab96c9&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=0c473678b281495f86f58b077be04e1b => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:36.420957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52110-7407-4668-9dcf-4410df3d6481 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:36.421378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52110-7407-4668-9dcf-4410df3d6481 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:36.421515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52110-7407-4668-9dcf-4410df3d6481 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:36.422919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 629/1261] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:49:36.427954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cd9a9-5f95-4a8a-880c-efcc4118e8fb None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:36.428390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cd9a9-5f95-4a8a-880c-efcc4118e8fb None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:36.428511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cd9a9-5f95-4a8a-880c-efcc4118e8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:36.444828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-018cd9a9-5f95-4a8a-880c-efcc4118e8fb None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:49:36.445990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-018cd9a9-5f95-4a8a-880c-efcc4118e8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:49:36.510449 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 633/1262] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:49:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:49:36.532521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d52110-7407-4668-9dcf-4410df3d6481 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:36.543676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:36.630180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d52110-7407-4668-9dcf-4410df3d6481 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:36.665231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8dea1-ac88-4fc0-8a9a-980c1927ce77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:36.665958 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8dea1-ac88-4fc0-8a9a-980c1927ce77 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:36.666244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8dea1-ac88-4fc0-8a9a-980c1927ce77 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:36.666558 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8dea1-ac88-4fc0-8a9a-980c1927ce77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:36.676690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8dea1-ac88-4fc0-8a9a-980c1927ce77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-63c8dea1-ac88-4fc0-8a9a-980c1927ce77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:49:36.684303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:36.685391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:36.686305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 630/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:49:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:36.691454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018cd9a9-5f95-4a8a-880c-efcc4118e8fb None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:36.709707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:36.787737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018cd9a9-5f95-4a8a-880c-efcc4118e8fb None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:36.820494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14985f3-1b5b-4ed0-bf22-de32f03f0a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:36.821146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14985f3-1b5b-4ed0-bf22-de32f03f0a1d None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:36.821431 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14985f3-1b5b-4ed0-bf22-de32f03f0a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:36.821707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14985f3-1b5b-4ed0-bf22-de32f03f0a1d None admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:36.822467 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14985f3-1b5b-4ed0-bf22-de32f03f0a1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:36.822777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14985f3-1b5b-4ed0-bf22-de32f03f0a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:36.824143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14985f3-1b5b-4ed0-bf22-de32f03f0a1d None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:36.831855 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 634/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:49:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:36.838211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8dea1-ac88-4fc0-8a9a-980c1927ce77 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:36.902112 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:36.934443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8dea1-ac88-4fc0-8a9a-980c1927ce77 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:36.973541 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc001ae-2683-41c8-a5e6-8b56b18630e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:36.974377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc001ae-2683-41c8-a5e6-8b56b18630e0 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:36.974661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc001ae-2683-41c8-a5e6-8b56b18630e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:36.974936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc001ae-2683-41c8-a5e6-8b56b18630e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:36.985342 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc001ae-2683-41c8-a5e6-8b56b18630e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:36.986802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc001ae-2683-41c8-a5e6-8b56b18630e0 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-6fc001ae-2683-41c8-a5e6-8b56b18630e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:37.003604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:37.004646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:37.005571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:37.005571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:37.005571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:37.005571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:37.005571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 631/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:37.011557 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14985f3-1b5b-4ed0-bf22-de32f03f0a1d None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:37.088889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:37.107590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14985f3-1b5b-4ed0-bf22-de32f03f0a1d None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:37.140440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33442267-aac2-4e92-b8a6-451fc04445fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:37.141085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442267-aac2-4e92-b8a6-451fc04445fc None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:37.141363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442267-aac2-4e92-b8a6-451fc04445fc None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:37.141633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442267-aac2-4e92-b8a6-451fc04445fc None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:37.142370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33442267-aac2-4e92-b8a6-451fc04445fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:37.142674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33442267-aac2-4e92-b8a6-451fc04445fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:37.144031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33442267-aac2-4e92-b8a6-451fc04445fc None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:37.155190 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 635/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:37.160783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc001ae-2683-41c8-a5e6-8b56b18630e0 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:37.268534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc001ae-2683-41c8-a5e6-8b56b18630e0 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:37.287416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:37.301411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aef8bc-0952-44fe-addb-d1742219fcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:37.302215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aef8bc-0952-44fe-addb-d1742219fcae None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:37.302530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aef8bc-0952-44fe-addb-d1742219fcae None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:37.302805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aef8bc-0952-44fe-addb-d1742219fcae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:37.310670 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aef8bc-0952-44fe-addb-d1742219fcae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:37.312114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aef8bc-0952-44fe-addb-d1742219fcae None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-d9aef8bc-0952-44fe-addb-d1742219fcae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:49:37.332462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:37.333653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:37.334886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:37.334886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:37.334886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:49:37.334886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:49:37.334886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:49:37.334886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:49:37.334886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:49:37.334886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:49:37.334886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:37.334886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:37.334886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 632/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:37.339433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33442267-aac2-4e92-b8a6-451fc04445fc None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:37.439564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33442267-aac2-4e92-b8a6-451fc04445fc None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:37.471572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f9112-3928-4866-b18d-8f1393bfb4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:37.472238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f9112-3928-4866-b18d-8f1393bfb4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:37.472522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f9112-3928-4866-b18d-8f1393bfb4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:37.472797 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f9112-3928-4866-b18d-8f1393bfb4d6 None admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:37.473623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f9112-3928-4866-b18d-8f1393bfb4d6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:37.473934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f9112-3928-4866-b18d-8f1393bfb4d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:37.475391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f9112-3928-4866-b18d-8f1393bfb4d6 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:37.484208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:37.493283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 636/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 12:49:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:37.498972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aef8bc-0952-44fe-addb-d1742219fcae None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:37.594080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aef8bc-0952-44fe-addb-d1742219fcae None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:37.626435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631c48f-ab8a-46f6-bce1-b786915cf9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:37.627214 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c48f-ab8a-46f6-bce1-b786915cf9ca None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:37.627500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c48f-ab8a-46f6-bce1-b786915cf9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:37.627772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c48f-ab8a-46f6-bce1-b786915cf9ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:37.638086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631c48f-ab8a-46f6-bce1-b786915cf9ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:37.639542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631c48f-ab8a-46f6-bce1-b786915cf9ca None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-3631c48f-ab8a-46f6-bce1-b786915cf9ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:37.655967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:37.657033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:37.658015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:37.658015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:37.658015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:37.658015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:37.658015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 633/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:37.662619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f9112-3928-4866-b18d-8f1393bfb4d6 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:37.759359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f9112-3928-4866-b18d-8f1393bfb4d6 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:37.791944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859e5c5-bc16-46f3-b6c9-c6619f5dae24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:37.792589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859e5c5-bc16-46f3-b6c9-c6619f5dae24 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:37.792869 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859e5c5-bc16-46f3-b6c9-c6619f5dae24 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:37.793141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859e5c5-bc16-46f3-b6c9-c6619f5dae24 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:37.793846 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859e5c5-bc16-46f3-b6c9-c6619f5dae24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:37.794148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859e5c5-bc16-46f3-b6c9-c6619f5dae24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:37.795532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859e5c5-bc16-46f3-b6c9-c6619f5dae24 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:37.806165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 637/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:37.811524 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631c48f-ab8a-46f6-bce1-b786915cf9ca None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:37.842356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:37.925200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631c48f-ab8a-46f6-bce1-b786915cf9ca None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:37.957065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebba17a-7a96-47dc-9077-bc7a430b9d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:37.957851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebba17a-7a96-47dc-9077-bc7a430b9d29 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:37.958134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebba17a-7a96-47dc-9077-bc7a430b9d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:37.958442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebba17a-7a96-47dc-9077-bc7a430b9d29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:37.965352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebba17a-7a96-47dc-9077-bc7a430b9d29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:37.966831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebba17a-7a96-47dc-9077-bc7a430b9d29 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-2ebba17a-7a96-47dc-9077-bc7a430b9d29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:49:37.981528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:37.982641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:37.983607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:49:37.983607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:49:37.983607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:49:37.983607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:49:37.983607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:49:37.983607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:49:37.983607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:37.983607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:37.983607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 634/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:49:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:37.988487 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859e5c5-bc16-46f3-b6c9-c6619f5dae24 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:38.044819 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:38.086100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859e5c5-bc16-46f3-b6c9-c6619f5dae24 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:38.117410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6687045-af79-474c-841c-65633bea525c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:38.118051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6687045-af79-474c-841c-65633bea525c None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:38.118351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6687045-af79-474c-841c-65633bea525c None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:38.118624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6687045-af79-474c-841c-65633bea525c None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:38.119397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6687045-af79-474c-841c-65633bea525c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:38.119702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6687045-af79-474c-841c-65633bea525c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:38.121130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6687045-af79-474c-841c-65633bea525c None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:38.133189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 638/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:49:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:38.138964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebba17a-7a96-47dc-9077-bc7a430b9d29 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:38.238042 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:38.250856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebba17a-7a96-47dc-9077-bc7a430b9d29 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:38.284643 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f5acc-b942-41ba-bf2d-8442d5d19124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:38.285524 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f5acc-b942-41ba-bf2d-8442d5d19124 None admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:38.285812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f5acc-b942-41ba-bf2d-8442d5d19124 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:38.286085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f5acc-b942-41ba-bf2d-8442d5d19124 None admin] PATH_INFO: `/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/630f0527bd844de9be072c459dab96c9/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:38.288049 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f5acc-b942-41ba-bf2d-8442d5d19124 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5887ec08efc449abba83b0c49e9f167, user_id=630f0527bd844de9be072c459dab96c9, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:38.289565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f5acc-b942-41ba-bf2d-8442d5d19124 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:38.301833 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 635/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 12:49:38 2022] PUT /identity/v3/projects/d5887ec08efc449abba83b0c49e9f167/users/630f0527bd844de9be072c459dab96c9/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:49:40.463881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabd625-85ff-45a5-956a-b98f7d708a27 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:40.464380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabd625-85ff-45a5-956a-b98f7d708a27 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:40.464466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabd625-85ff-45a5-956a-b98f7d708a27 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:40.465824 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 639/1274] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:49:40.470906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a7d0a-330f-415d-bfe5-80b63b94af4b None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:40.471292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a7d0a-330f-415d-bfe5-80b63b94af4b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:40.471466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a7d0a-330f-415d-bfe5-80b63b94af4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:40.488248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-d71a7d0a-330f-415d-bfe5-80b63b94af4b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:49:40.489424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d71a7d0a-330f-415d-bfe5-80b63b94af4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:49:40.552971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 636/1275] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:49:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:49:40.576095 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabd625-85ff-45a5-956a-b98f7d708a27 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:40.674521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabd625-85ff-45a5-956a-b98f7d708a27 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:40.691805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:40.708210 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcec44-e549-4c9f-8af1-a7a80393775f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:40.709014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcec44-e549-4c9f-8af1-a7a80393775f None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:40.709297 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcec44-e549-4c9f-8af1-a7a80393775f None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:40.709567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcec44-e549-4c9f-8af1-a7a80393775f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:40.720068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fcec44-e549-4c9f-8af1-a7a80393775f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-c4fcec44-e549-4c9f-8af1-a7a80393775f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 12:49:40.727770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:40.728856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:40.729886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 640/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 12:49:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:40.734735 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a7d0a-330f-415d-bfe5-80b63b94af4b None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:40.812287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:40.831177 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a7d0a-330f-415d-bfe5-80b63b94af4b None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:40.862967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67137472-cd23-44f4-a9d5-6ca28bc6ebb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:40.863625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67137472-cd23-44f4-a9d5-6ca28bc6ebb5 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:40.863913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67137472-cd23-44f4-a9d5-6ca28bc6ebb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:40.864189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67137472-cd23-44f4-a9d5-6ca28bc6ebb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:40.864930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67137472-cd23-44f4-a9d5-6ca28bc6ebb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:40.865236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67137472-cd23-44f4-a9d5-6ca28bc6ebb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:40.867140 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67137472-cd23-44f4-a9d5-6ca28bc6ebb5 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:40.874858 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 637/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 12:49:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:40.880498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcec44-e549-4c9f-8af1-a7a80393775f None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:40.977504 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcec44-e549-4c9f-8af1-a7a80393775f None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:41.009918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc6979-c162-45f8-a891-0bffe0fdcb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:41.010793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc6979-c162-45f8-a891-0bffe0fdcb70 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:41.011080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc6979-c162-45f8-a891-0bffe0fdcb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:41.011358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc6979-c162-45f8-a891-0bffe0fdcb70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:41.021509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc6979-c162-45f8-a891-0bffe0fdcb70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:41.023501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc6979-c162-45f8-a891-0bffe0fdcb70 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-efcc6979-c162-45f8-a891-0bffe0fdcb70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:41.040488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:41.041528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:41.042661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:41.042661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:41.042661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:41.042661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:41.042661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 641/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:41.047164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67137472-cd23-44f4-a9d5-6ca28bc6ebb5 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:41.142795 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67137472-cd23-44f4-a9d5-6ca28bc6ebb5 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:41.174515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eff0af-a0c3-4005-b1da-8aee52e347b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:41.175160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eff0af-a0c3-4005-b1da-8aee52e347b3 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:41.175442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eff0af-a0c3-4005-b1da-8aee52e347b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:41.175724 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eff0af-a0c3-4005-b1da-8aee52e347b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:41.176578 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eff0af-a0c3-4005-b1da-8aee52e347b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:41.176888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eff0af-a0c3-4005-b1da-8aee52e347b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:41.178225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eff0af-a0c3-4005-b1da-8aee52e347b3 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:41.191746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 638/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:41.200066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc6979-c162-45f8-a891-0bffe0fdcb70 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:41.238011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:41.307277 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc6979-c162-45f8-a891-0bffe0fdcb70 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:41.340385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8bae6-3e93-4999-9aad-cfef5ad4bb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:41.341184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8bae6-3e93-4999-9aad-cfef5ad4bb4b None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:41.341466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8bae6-3e93-4999-9aad-cfef5ad4bb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:41.341740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8bae6-3e93-4999-9aad-cfef5ad4bb4b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:41.349054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8bae6-3e93-4999-9aad-cfef5ad4bb4b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:41.350523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8bae6-3e93-4999-9aad-cfef5ad4bb4b None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-17f8bae6-3e93-4999-9aad-cfef5ad4bb4b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 12:49:41.364161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:41.365284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:41.366536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:41.366536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:41.366536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 12:49:41.366536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 12:49:41.366536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 12:49:41.366536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 12:49:41.366536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 12:49:41.366536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 12:49:41.366536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 12:49:41.366536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:41.366536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 642/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:41.371315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eff0af-a0c3-4005-b1da-8aee52e347b3 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:41.375005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:41.470904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eff0af-a0c3-4005-b1da-8aee52e347b3 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:41.504097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ebf70-2af4-4a39-afe7-9730ac8825d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:41.504696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ebf70-2af4-4a39-afe7-9730ac8825d6 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:41.504980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ebf70-2af4-4a39-afe7-9730ac8825d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:41.505257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ebf70-2af4-4a39-afe7-9730ac8825d6 None admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:41.506363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ebf70-2af4-4a39-afe7-9730ac8825d6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:41.506673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ebf70-2af4-4a39-afe7-9730ac8825d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:41.508649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ebf70-2af4-4a39-afe7-9730ac8825d6 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:41.524947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 639/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 12:49:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 12:49:41.530649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8bae6-3e93-4999-9aad-cfef5ad4bb4b None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:41.617159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:41.627034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8bae6-3e93-4999-9aad-cfef5ad4bb4b None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:41.659256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587cb7f-e2b3-461c-b8e0-787b8bac3af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:41.659882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587cb7f-e2b3-461c-b8e0-787b8bac3af2 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:41.660162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587cb7f-e2b3-461c-b8e0-787b8bac3af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:41.660434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587cb7f-e2b3-461c-b8e0-787b8bac3af2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:41.724730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587cb7f-e2b3-461c-b8e0-787b8bac3af2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:41.726189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587cb7f-e2b3-461c-b8e0-787b8bac3af2 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:41.810457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:41.821157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 643/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 12:49:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 12:49:41.826898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ebf70-2af4-4a39-afe7-9730ac8825d6 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:41.885790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:41.923192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ebf70-2af4-4a39-afe7-9730ac8825d6 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:41.955660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb0814-df26-4814-b5a6-3d6c7330fa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:41.956437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb0814-df26-4814-b5a6-3d6c7330fa9e None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:41.956720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb0814-df26-4814-b5a6-3d6c7330fa9e None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:41.956996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb0814-df26-4814-b5a6-3d6c7330fa9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:41.967161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb0814-df26-4814-b5a6-3d6c7330fa9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:41.968582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb0814-df26-4814-b5a6-3d6c7330fa9e None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-7ecb0814-df26-4814-b5a6-3d6c7330fa9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 12:49:41.985427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:41.986477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:41.987531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 12:49:41.987531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 12:49:41.987531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 12:49:41.987531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:41.987531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 640/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:41.992114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587cb7f-e2b3-461c-b8e0-787b8bac3af2 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:42.087673 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587cb7f-e2b3-461c-b8e0-787b8bac3af2 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:42.119743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c81fb-a943-4f18-a2cf-0d03d62d3e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:42.120390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c81fb-a943-4f18-a2cf-0d03d62d3e86 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:42.120675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c81fb-a943-4f18-a2cf-0d03d62d3e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:42.120954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c81fb-a943-4f18-a2cf-0d03d62d3e86 None admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:42.121670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c81fb-a943-4f18-a2cf-0d03d62d3e86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:42.121977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c81fb-a943-4f18-a2cf-0d03d62d3e86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:42.123374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c81fb-a943-4f18-a2cf-0d03d62d3e86 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:42.134938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 644/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 12:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:42.140230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb0814-df26-4814-b5a6-3d6c7330fa9e None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:42.236635 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb0814-df26-4814-b5a6-3d6c7330fa9e None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:42.250492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:42.271247 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d4eb0-731a-4d17-9025-538d4c4b150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:42.272047 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4eb0-731a-4d17-9025-538d4c4b150c None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:42.272637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4eb0-731a-4d17-9025-538d4c4b150c None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:42.272916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4eb0-731a-4d17-9025-538d4c4b150c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:42.279466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d4eb0-731a-4d17-9025-538d4c4b150c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:42.280889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d4eb0-731a-4d17-9025-538d4c4b150c None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application [None req-105d4eb0-731a-4d17-9025-538d4c4b150c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 12:49:42.294213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application with Lock( May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:42.295268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:42.296228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 12:49:42.296228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 12:49:42.296228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 12:49:42.296228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 12:49:42.296228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 12:49:42.296228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 12:49:42.296228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 12:49:42.296228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: ERROR keystone.server.flask.application May 24 12:49:42.296228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 641/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 12:49:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:42.301149 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c81fb-a943-4f18-a2cf-0d03d62d3e86 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:42.349724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:42.399070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c81fb-a943-4f18-a2cf-0d03d62d3e86 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:42.431661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a311b-4e1d-4bf9-a6d6-da633e44e1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:42.432307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a311b-4e1d-4bf9-a6d6-da633e44e1bf None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:42.432587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a311b-4e1d-4bf9-a6d6-da633e44e1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:42.432856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a311b-4e1d-4bf9-a6d6-da633e44e1bf None admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:42.433620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a311b-4e1d-4bf9-a6d6-da633e44e1bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:42.433924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a311b-4e1d-4bf9-a6d6-da633e44e1bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:42.435365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a311b-4e1d-4bf9-a6d6-da633e44e1bf None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:42.447359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 645/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 12:49:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:42.453250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d4eb0-731a-4d17-9025-538d4c4b150c None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:42.547529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d4eb0-731a-4d17-9025-538d4c4b150c None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:42.579661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b28ec8-37f0-4012-ae63-bf24f68ede14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:42.580294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b28ec8-37f0-4012-ae63-bf24f68ede14 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:42.580577 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b28ec8-37f0-4012-ae63-bf24f68ede14 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:42.580851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b28ec8-37f0-4012-ae63-bf24f68ede14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:42.581664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b28ec8-37f0-4012-ae63-bf24f68ede14 None admin] RBAC: Adding query filter params (role.id=0c473678b281495f86f58b077be04e1b, scope.project.id=d5887ec08efc449abba83b0c49e9f167, user.id=630f0527bd844de9be072c459dab96c9) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:42.581971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b28ec8-37f0-4012-ae63-bf24f68ede14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:42.583409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b28ec8-37f0-4012-ae63-bf24f68ede14 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:42.589009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 642/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 12:49:42 2022] GET /identity/v3/role_assignments?user.id=630f0527bd844de9be072c459dab96c9&scope.project.id=d5887ec08efc449abba83b0c49e9f167&role.id=0c473678b281495f86f58b077be04e1b => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:44.751038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55add9f-1731-4115-8702-57c6075c6fb8 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:44.751535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55add9f-1731-4115-8702-57c6075c6fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:44.751535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55add9f-1731-4115-8702-57c6075c6fb8 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:44.752884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 646/1288] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:49:44.758197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449aa4d-e6a6-4546-9b71-f1b91c5152d4 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:44.758593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449aa4d-e6a6-4546-9b71-f1b91c5152d4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:44.758788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449aa4d-e6a6-4546-9b71-f1b91c5152d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:44.775594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7449aa4d-e6a6-4546-9b71-f1b91c5152d4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:49:44.776752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7449aa4d-e6a6-4546-9b71-f1b91c5152d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:49:44.805115 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:44.841339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 643/1289] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:49:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:49:44.862722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55add9f-1731-4115-8702-57c6075c6fb8 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:44.959728 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55add9f-1731-4115-8702-57c6075c6fb8 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:44.970351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:44.993059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05086b-157b-47aa-a409-c62359f58522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:44.993835 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05086b-157b-47aa-a409-c62359f58522 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:44.994119 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05086b-157b-47aa-a409-c62359f58522 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:44.994426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05086b-157b-47aa-a409-c62359f58522 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:44.995085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05086b-157b-47aa-a409-c62359f58522 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:44.996454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05086b-157b-47aa-a409-c62359f58522 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-3c05086b-157b-47aa-a409-c62359f58522 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:45.008890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:45.010078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:45.010989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 647/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 12:49:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:45.015891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449aa4d-e6a6-4546-9b71-f1b91c5152d4 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:45.115395 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449aa4d-e6a6-4546-9b71-f1b91c5152d4 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:45.148992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a322f3-162f-4f6c-96ca-a8e0ff745edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:45.149644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a322f3-162f-4f6c-96ca-a8e0ff745edb None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:45.149926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a322f3-162f-4f6c-96ca-a8e0ff745edb None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:45.150197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a322f3-162f-4f6c-96ca-a8e0ff745edb None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:45.150921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a322f3-162f-4f6c-96ca-a8e0ff745edb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:45.151223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a322f3-162f-4f6c-96ca-a8e0ff745edb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:45.152579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a322f3-162f-4f6c-96ca-a8e0ff745edb None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:45.157224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 644/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Tue May 24 12:49:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:45.162943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05086b-157b-47aa-a409-c62359f58522 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:45.269214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05086b-157b-47aa-a409-c62359f58522 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:45.300845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adb8df-417f-4150-adbc-0c81dd01ec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:45.301502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb8df-417f-4150-adbc-0c81dd01ec73 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:45.301786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb8df-417f-4150-adbc-0c81dd01ec73 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:45.302061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb8df-417f-4150-adbc-0c81dd01ec73 None admin] PATH_INFO: `/v3/services` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:45.302810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adb8df-417f-4150-adbc-0c81dd01ec73 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:45.303118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adb8df-417f-4150-adbc-0c81dd01ec73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:45.304989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adb8df-417f-4150-adbc-0c81dd01ec73 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:45.309379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 648/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Tue May 24 12:49:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:47.438677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba51e9-e63f-4776-a3b4-93445aa98114 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:47.439165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba51e9-e63f-4776-a3b4-93445aa98114 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:47.439253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba51e9-e63f-4776-a3b4-93445aa98114 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:47.440587 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 645/1293] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:49:47.446316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee49cf-f8b5-4d41-b76f-48ec7ea107d7 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:47.446760 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee49cf-f8b5-4d41-b76f-48ec7ea107d7 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:47.446851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee49cf-f8b5-4d41-b76f-48ec7ea107d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:47.463952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6cee49cf-f8b5-4d41-b76f-48ec7ea107d7 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:49:47.465123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6cee49cf-f8b5-4d41-b76f-48ec7ea107d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:49:47.493408 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:47.529131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 649/1294] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:49:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:49:47.550960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba51e9-e63f-4776-a3b4-93445aa98114 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:47.581827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:47.650225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba51e9-e63f-4776-a3b4-93445aa98114 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:47.689825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1174b-e8cb-4765-84df-7b1283fc4a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:47.690444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1174b-e8cb-4765-84df-7b1283fc4a4a None admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:47.690730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1174b-e8cb-4765-84df-7b1283fc4a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:47.691007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1174b-e8cb-4765-84df-7b1283fc4a4a None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:47.691729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1174b-e8cb-4765-84df-7b1283fc4a4a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:47.693088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1174b-e8cb-4765-84df-7b1283fc4a4a None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:47.706392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 646/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 12:49:47 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:49:49.890127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510902ba-55e9-4892-b93b-c1d9e3056041 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:49.890701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510902ba-55e9-4892-b93b-c1d9e3056041 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:49.890796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510902ba-55e9-4892-b93b-c1d9e3056041 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:49.892103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 650/1296] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:49:49.897785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b387329-df8b-4bff-803d-04694583162e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:49.898158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b387329-df8b-4bff-803d-04694583162e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:49.898365 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b387329-df8b-4bff-803d-04694583162e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:49.915300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-3b387329-df8b-4bff-803d-04694583162e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:49:49.916501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3b387329-df8b-4bff-803d-04694583162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:49:49.985913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 647/1297] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:49:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 12:49:50.008081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510902ba-55e9-4892-b93b-c1d9e3056041 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:50.106366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510902ba-55e9-4892-b93b-c1d9e3056041 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:50.139298 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e39e46-1ad6-4526-86f4-c2b2324d1308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:50.140074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e39e46-1ad6-4526-86f4-c2b2324d1308 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:50.140406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e39e46-1ad6-4526-86f4-c2b2324d1308 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:50.140636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e39e46-1ad6-4526-86f4-c2b2324d1308 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:50.141279 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e39e46-1ad6-4526-86f4-c2b2324d1308 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:50.142667 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e39e46-1ad6-4526-86f4-c2b2324d1308 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-75e39e46-1ad6-4526-86f4-c2b2324d1308 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:50.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:50.155385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:50.156391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 651/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 12:49:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:50.161158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b387329-df8b-4bff-803d-04694583162e None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:50.179726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:50.271354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b387329-df8b-4bff-803d-04694583162e None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:50.304446 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db8dda-5ae1-4cc3-8fa4-96b3f4234f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:50.305097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db8dda-5ae1-4cc3-8fa4-96b3f4234f41 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:50.305378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db8dda-5ae1-4cc3-8fa4-96b3f4234f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:50.305653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db8dda-5ae1-4cc3-8fa4-96b3f4234f41 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:50.306379 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db8dda-5ae1-4cc3-8fa4-96b3f4234f41 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:50.306692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db8dda-5ae1-4cc3-8fa4-96b3f4234f41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:50.308050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db8dda-5ae1-4cc3-8fa4-96b3f4234f41 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:50.313037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 648/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Tue May 24 12:49:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:50.318988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e39e46-1ad6-4526-86f4-c2b2324d1308 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:50.353198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:50.420454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e39e46-1ad6-4526-86f4-c2b2324d1308 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:50.453481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c2067-05f6-4c9f-bf14-6142e593b0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:50.454136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c2067-05f6-4c9f-bf14-6142e593b0ff None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:50.454532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c2067-05f6-4c9f-bf14-6142e593b0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:50.454816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c2067-05f6-4c9f-bf14-6142e593b0ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:50.455603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c2067-05f6-4c9f-bf14-6142e593b0ff None admin] RBAC: Adding query filter params (interface=public, service_id=0872cbc2467b4d3ebbb0a45f3ba5400f, region_id=RegionOne) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:50.455916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c2067-05f6-4c9f-bf14-6142e593b0ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:50.457288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c2067-05f6-4c9f-bf14-6142e593b0ff None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:50.462209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 652/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 12:49:50 2022] GET /identity/v3/endpoints?service_id=0872cbc2467b4d3ebbb0a45f3ba5400f&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:50.467699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db8dda-5ae1-4cc3-8fa4-96b3f4234f41 None admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:50.545223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:50.563845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db8dda-5ae1-4cc3-8fa4-96b3f4234f41 None admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:50.596619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13500faf-31ee-4dcb-a9bb-fde899d4cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:50.597273 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13500faf-31ee-4dcb-a9bb-fde899d4cb6f None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:50.597557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13500faf-31ee-4dcb-a9bb-fde899d4cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:50.597834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13500faf-31ee-4dcb-a9bb-fde899d4cb6f None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:50.598540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13500faf-31ee-4dcb-a9bb-fde899d4cb6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:50.599896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13500faf-31ee-4dcb-a9bb-fde899d4cb6f None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:50.606587 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 649/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 12:49:50 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:49:52.745215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49b51e-835b-4a98-aee6-011186dd09b0 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:52.745724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49b51e-835b-4a98-aee6-011186dd09b0 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:52.745724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49b51e-835b-4a98-aee6-011186dd09b0 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:52.747090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 653/1302] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:49:52.752181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91dce3-4fae-4a7b-b02b-849505538213 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:52.752602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91dce3-4fae-4a7b-b02b-849505538213 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:52.752741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91dce3-4fae-4a7b-b02b-849505538213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:52.769476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1d91dce3-4fae-4a7b-b02b-849505538213 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:49:52.770703 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1d91dce3-4fae-4a7b-b02b-849505538213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:49:52.837032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 650/1303] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:49:52 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 12:49:52.859714 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49b51e-835b-4a98-aee6-011186dd09b0 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:52.955300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:52.962765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49b51e-835b-4a98-aee6-011186dd09b0 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:52.995346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47616dd4-f721-496f-9cf6-3e97fdc86c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:52.996076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47616dd4-f721-496f-9cf6-3e97fdc86c36 None admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:52.996359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47616dd4-f721-496f-9cf6-3e97fdc86c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:52.996633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47616dd4-f721-496f-9cf6-3e97fdc86c36 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:52.997278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47616dd4-f721-496f-9cf6-3e97fdc86c36 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:52.998675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47616dd4-f721-496f-9cf6-3e97fdc86c36 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application [None req-47616dd4-f721-496f-9cf6-3e97fdc86c36 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 12:49:53.010538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 12:49:53.011711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.get_or_create( May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application with Lock( May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self._enter() May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application return self.creator() May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application created_value = creator( May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: ERROR keystone.server.flask.application May 24 12:49:53.012830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 654/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 12:49:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 12:49:53.017668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91dce3-4fae-4a7b-b02b-849505538213 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:53.091837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:49:53.114096 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91dce3-4fae-4a7b-b02b-849505538213 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:53.147724 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5be783-68b5-4c3d-ac68-ebc8939a5710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:53.148376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5be783-68b5-4c3d-ac68-ebc8939a5710 None admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:53.148663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5be783-68b5-4c3d-ac68-ebc8939a5710 None admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:53.148948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5be783-68b5-4c3d-ac68-ebc8939a5710 None admin] PATH_INFO: `/v3/services` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:53.149653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5be783-68b5-4c3d-ac68-ebc8939a5710 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:49:53.149963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5be783-68b5-4c3d-ac68-ebc8939a5710 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:53.151360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5be783-68b5-4c3d-ac68-ebc8939a5710 None admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:53.156326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 651/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Tue May 24 12:49:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:49:53.161858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47616dd4-f721-496f-9cf6-3e97fdc86c36 None admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:49:53.258560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47616dd4-f721-496f-9cf6-3e97fdc86c36 None admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:49:53.290745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01e54f-6c27-4fda-8631-8f870123ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:49:53.291380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01e54f-6c27-4fda-8631-8f870123ddf4 None admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:49:53.291666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01e54f-6c27-4fda-8631-8f870123ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:49:53.291944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01e54f-6c27-4fda-8631-8f870123ddf4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:49:53.292687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01e54f-6c27-4fda-8631-8f870123ddf4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:49:53.294048 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01e54f-6c27-4fda-8631-8f870123ddf4 None admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:49:53.311375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 655/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 12:49:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 12:50:03.153793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=30 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:50:03.307026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:50:56.645672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecdf26-d418-4fb8-a5ba-a585eb357c28 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:50:56.647402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecdf26-d418-4fb8-a5ba-a585eb357c28 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:50:56.648440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecdf26-d418-4fb8-a5ba-a585eb357c28 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:50:56.653663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 652/1307] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:50:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:50:56.657091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574cea-a56d-4b1a-bdcb-49f30e70efd7 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:50:56.657507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574cea-a56d-4b1a-bdcb-49f30e70efd7 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:50:56.657598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574cea-a56d-4b1a-bdcb-49f30e70efd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:50:56.837685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-fb574cea-a56d-4b1a-bdcb-49f30e70efd7 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:50:56.839063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-fb574cea-a56d-4b1a-bdcb-49f30e70efd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:50:56.955881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 656/1308] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:50:57.005497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19857b88-c359-4952-ae66-d703c94525cb None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:50:57.005781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19857b88-c359-4952-ae66-d703c94525cb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:50:57.006088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19857b88-c359-4952-ae66-d703c94525cb None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:50:57.007509 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 653/1309] 10.222.0.45 () {54 vars in 964 bytes} [Tue May 24 12:50:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:50:57.013509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d370425-baf6-4760-94ea-af0ac0882f21 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:50:57.013805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d370425-baf6-4760-94ea-af0ac0882f21 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:50:57.014080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d370425-baf6-4760-94ea-af0ac0882f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:50:57.100978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7d370425-baf6-4760-94ea-af0ac0882f21 None None] MFA Rules not processed for user `bb111be6bc2946d3b50b8d99a89cf8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:50:57.102212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7d370425-baf6-4760-94ea-af0ac0882f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:50:57.124824 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:50:57.184113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 657/1310] 10.222.0.45 () {58 vars in 1061 bytes} [Tue May 24 12:50:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:50:57.193152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19857b88-c359-4952-ae66-d703c94525cb None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:50:57.194930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-19857b88-c359-4952-ae66-d703c94525cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:50:57.324882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19857b88-c359-4952-ae66-d703c94525cb None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:50:57.408513 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd4b4b-1904-4256-b594-1077012e2d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:50:57.409186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4b4b-1904-4256-b594-1077012e2d68 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:50:57.409473 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4b4b-1904-4256-b594-1077012e2d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:50:57.409753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4b4b-1904-4256-b594-1077012e2d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:50:57.412076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6fdd4b4b-1904-4256-b594-1077012e2d68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:50:57.475766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:50:57.543550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd4b4b-1904-4256-b594-1077012e2d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:50:57.545008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd4b4b-1904-4256-b594-1077012e2d68 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:50:57.565742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 654/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:50:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:00.163237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936abde-9569-4de4-9bf3-6e7a4632a705 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:00.163811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936abde-9569-4de4-9bf3-6e7a4632a705 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:00.163811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936abde-9569-4de4-9bf3-6e7a4632a705 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:00.165178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 658/1312] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:00.170498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0745-e59f-40fb-9503-1e9564bb6e0c None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:00.170955 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0745-e59f-40fb-9503-1e9564bb6e0c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:00.171263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0745-e59f-40fb-9503-1e9564bb6e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:00.255143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-fc8f0745-e59f-40fb-9503-1e9564bb6e0c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:00.256516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-fc8f0745-e59f-40fb-9503-1e9564bb6e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:00.366318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 655/1313] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:00.402418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936abde-9569-4de4-9bf3-6e7a4632a705 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:00.402418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3936abde-9569-4de4-9bf3-6e7a4632a705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:00.510545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936abde-9569-4de4-9bf3-6e7a4632a705 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:00.587564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:00.590613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fe760-4f6f-49ef-a9eb-fffc4e6e003a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:00.591653 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fe760-4f6f-49ef-a9eb-fffc4e6e003a service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:00.592491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fe760-4f6f-49ef-a9eb-fffc4e6e003a service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:00.592886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fe760-4f6f-49ef-a9eb-fffc4e6e003a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:00.594965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a46fe760-4f6f-49ef-a9eb-fffc4e6e003a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:00.714644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fe760-4f6f-49ef-a9eb-fffc4e6e003a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:00.716011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fe760-4f6f-49ef-a9eb-fffc4e6e003a service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:00.734894 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 659/1314] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:03.291776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6bf2-051d-45b1-b6f0-cb1b20c9b0d2 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:03.292313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6bf2-051d-45b1-b6f0-cb1b20c9b0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:03.292313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6bf2-051d-45b1-b6f0-cb1b20c9b0d2 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:03.293726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 656/1315] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:03.299075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfe824-0b11-4698-b0bb-f1e5a0a7fb81 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:03.299499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfe824-0b11-4698-b0bb-f1e5a0a7fb81 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:03.299636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfe824-0b11-4698-b0bb-f1e5a0a7fb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:03.364040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e3bfe824-0b11-4698-b0bb-f1e5a0a7fb81 None None] MFA Rules not processed for user `944f4c75142549088d2481820e193ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:03.365234 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e3bfe824-0b11-4698-b0bb-f1e5a0a7fb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:03.416013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:03.460649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 660/1316] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:03 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:03.499210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af6bf2-051d-45b1-b6f0-cb1b20c9b0d2 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:03.501175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a5af6bf2-051d-45b1-b6f0-cb1b20c9b0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:03.512024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:03.607030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af6bf2-051d-45b1-b6f0-cb1b20c9b0d2 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:03.679197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36673a-88ae-4beb-b36d-77d2cf16c995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:03.679840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36673a-88ae-4beb-b36d-77d2cf16c995 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:03.680122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36673a-88ae-4beb-b36d-77d2cf16c995 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:03.680396 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36673a-88ae-4beb-b36d-77d2cf16c995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:03.682024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ec36673a-88ae-4beb-b36d-77d2cf16c995 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:03.742628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:03.789720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36673a-88ae-4beb-b36d-77d2cf16c995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:03.791188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36673a-88ae-4beb-b36d-77d2cf16c995 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:03.809638 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 657/1317] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:03 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:06.572259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f2556-db56-47c2-b120-d7a61b3fd5e2 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:06.572805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f2556-db56-47c2-b120-d7a61b3fd5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:06.572805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f2556-db56-47c2-b120-d7a61b3fd5e2 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:06.574212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 661/1318] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:06.579520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ccc1d-d9dc-414c-b7c6-5ec28847a1d0 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:06.579950 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ccc1d-d9dc-414c-b7c6-5ec28847a1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:06.580089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ccc1d-d9dc-414c-b7c6-5ec28847a1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:06.643409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e75ccc1d-d9dc-414c-b7c6-5ec28847a1d0 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:06.645180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e75ccc1d-d9dc-414c-b7c6-5ec28847a1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:06.739505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 658/1319] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:06.777860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f2556-db56-47c2-b120-d7a61b3fd5e2 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:06.779408 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-572f2556-db56-47c2-b120-d7a61b3fd5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:06.883597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f2556-db56-47c2-b120-d7a61b3fd5e2 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:06.955869 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4553bf-12c5-4473-af58-f044ebd190e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:06.956518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4553bf-12c5-4473-af58-f044ebd190e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:06.956802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4553bf-12c5-4473-af58-f044ebd190e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:06.957118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4553bf-12c5-4473-af58-f044ebd190e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:06.958777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7e4553bf-12c5-4473-af58-f044ebd190e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:06.966055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:07.065774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4553bf-12c5-4473-af58-f044ebd190e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:07.067795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4553bf-12c5-4473-af58-f044ebd190e7 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:07.085591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 662/1320] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:09.974361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf80c02-014d-40c4-a53d-aac600a884c3 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:09.974920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf80c02-014d-40c4-a53d-aac600a884c3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:09.974920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf80c02-014d-40c4-a53d-aac600a884c3 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:09.976301 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 659/1321] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:09.981498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a23a08-7557-460e-9a53-e4994c02af4e None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:09.981899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a23a08-7557-460e-9a53-e4994c02af4e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:09.982052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a23a08-7557-460e-9a53-e4994c02af4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:10.046212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-85a23a08-7557-460e-9a53-e4994c02af4e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:10.047430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-85a23a08-7557-460e-9a53-e4994c02af4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:10.103683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:10.142146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 663/1322] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:10.177319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf80c02-014d-40c4-a53d-aac600a884c3 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:10.178901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5bf80c02-014d-40c4-a53d-aac600a884c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:10.200967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:10.299350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf80c02-014d-40c4-a53d-aac600a884c3 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:10.372819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ce186-fde6-46e2-8a86-5fefc729f19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:10.373467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ce186-fde6-46e2-8a86-5fefc729f19d service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:10.373751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ce186-fde6-46e2-8a86-5fefc729f19d service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:10.374028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ce186-fde6-46e2-8a86-5fefc729f19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:10.375681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a20ce186-fde6-46e2-8a86-5fefc729f19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:10.447547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:10.487411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ce186-fde6-46e2-8a86-5fefc729f19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:10.488938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ce186-fde6-46e2-8a86-5fefc729f19d service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:10.507532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 660/1323] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:12.763229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d52b65-ee2b-472d-b557-108622c2b77a None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:12.763733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d52b65-ee2b-472d-b557-108622c2b77a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:12.763733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d52b65-ee2b-472d-b557-108622c2b77a None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:12.765062 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 664/1324] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:12.770467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52a2a6-25e6-421f-b966-fdd3bbe7f05e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:12.770859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52a2a6-25e6-421f-b966-fdd3bbe7f05e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:12.771026 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52a2a6-25e6-421f-b966-fdd3bbe7f05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:12.833822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-3c52a2a6-25e6-421f-b966-fdd3bbe7f05e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:12.835034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3c52a2a6-25e6-421f-b966-fdd3bbe7f05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:12.932765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 661/1325] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:12.968194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d52b65-ee2b-472d-b557-108622c2b77a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:12.969718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-89d52b65-ee2b-472d-b557-108622c2b77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:13.075649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d52b65-ee2b-472d-b557-108622c2b77a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:13.148230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fc3f5-f52b-4aec-9360-5ec93a74d48e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:13.148886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fc3f5-f52b-4aec-9360-5ec93a74d48e service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:13.149175 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fc3f5-f52b-4aec-9360-5ec93a74d48e service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:13.149455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fc3f5-f52b-4aec-9360-5ec93a74d48e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:13.151116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-812fc3f5-f52b-4aec-9360-5ec93a74d48e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:13.167390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:13.262007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fc3f5-f52b-4aec-9360-5ec93a74d48e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:13.263483 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fc3f5-f52b-4aec-9360-5ec93a74d48e service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:13.281031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 665/1326] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:15.522379 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d705d-6ed5-492a-8b22-057940b22550 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:15.522936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d705d-6ed5-492a-8b22-057940b22550 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:15.522936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d705d-6ed5-492a-8b22-057940b22550 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:15.524310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 662/1327] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:15.529555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45992844-085d-41bd-931f-8592db495cc4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:15.529979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45992844-085d-41bd-931f-8592db495cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:15.530110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45992844-085d-41bd-931f-8592db495cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:15.593513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-45992844-085d-41bd-931f-8592db495cc4 None None] MFA Rules not processed for user `944f4c75142549088d2481820e193ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:15.594736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-45992844-085d-41bd-931f-8592db495cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:15.657914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:15.691681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 666/1328] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:15.729053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d705d-6ed5-492a-8b22-057940b22550 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:15.730639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b63d705d-6ed5-492a-8b22-057940b22550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:15.761172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:15.834777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d705d-6ed5-492a-8b22-057940b22550 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:15.908665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4f305-c6ca-41dd-bc82-1759534edea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:15.909352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4f305-c6ca-41dd-bc82-1759534edea5 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:15.909645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4f305-c6ca-41dd-bc82-1759534edea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:15.910092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4f305-c6ca-41dd-bc82-1759534edea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:15.911785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-39a4f305-c6ca-41dd-bc82-1759534edea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:15.987303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:16.020028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4f305-c6ca-41dd-bc82-1759534edea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:16.021467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4f305-c6ca-41dd-bc82-1759534edea5 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:16.039072 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 663/1329] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:15 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:18.495561 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870c25d-351d-4caa-be0a-ecc5bfe1c401 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:18.496041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870c25d-351d-4caa-be0a-ecc5bfe1c401 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:18.496041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870c25d-351d-4caa-be0a-ecc5bfe1c401 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:18.497419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 667/1330] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:18.502707 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e3eaa-6326-4c3a-8532-c3c9250a9fb0 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:18.503094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e3eaa-6326-4c3a-8532-c3c9250a9fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:18.503455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e3eaa-6326-4c3a-8532-c3c9250a9fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:18.570841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-670e3eaa-6326-4c3a-8532-c3c9250a9fb0 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:18.572035 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-670e3eaa-6326-4c3a-8532-c3c9250a9fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:18.669714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 664/1331] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:18.704816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870c25d-351d-4caa-be0a-ecc5bfe1c401 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:18.706533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f870c25d-351d-4caa-be0a-ecc5bfe1c401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:18.813181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870c25d-351d-4caa-be0a-ecc5bfe1c401 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:18.890792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfcd69e-cdc0-410d-8016-29fada7bc0fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:18.891450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcd69e-cdc0-410d-8016-29fada7bc0fd service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:18.891728 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcd69e-cdc0-410d-8016-29fada7bc0fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:18.891998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcd69e-cdc0-410d-8016-29fada7bc0fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:18.893636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5cfcd69e-cdc0-410d-8016-29fada7bc0fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:18.924107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:19.020153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfcd69e-cdc0-410d-8016-29fada7bc0fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:19.021617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfcd69e-cdc0-410d-8016-29fada7bc0fd service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:19.041349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 668/1332] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:21.283320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3452f-91c3-4ff1-ad5c-4282f3ca8100 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:21.283900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3452f-91c3-4ff1-ad5c-4282f3ca8100 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:21.283900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3452f-91c3-4ff1-ad5c-4282f3ca8100 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:21.285296 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 665/1333] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:21.290371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cdf59-93d3-4c84-8843-753ae0bbe431 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:21.290783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cdf59-93d3-4c84-8843-753ae0bbe431 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:21.290944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cdf59-93d3-4c84-8843-753ae0bbe431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:21.355837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9b0cdf59-93d3-4c84-8843-753ae0bbe431 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:21.357032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9b0cdf59-93d3-4c84-8843-753ae0bbe431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:21.430678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:21.452691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 669/1334] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:21.493343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3452f-91c3-4ff1-ad5c-4282f3ca8100 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:21.494887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-0bd3452f-91c3-4ff1-ad5c-4282f3ca8100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:21.534498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:21.595506 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3452f-91c3-4ff1-ad5c-4282f3ca8100 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:21.666820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ac80c-7627-47c3-8786-0f48870ff5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:21.667466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ac80c-7627-47c3-8786-0f48870ff5ca service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:21.667752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ac80c-7627-47c3-8786-0f48870ff5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:21.668028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ac80c-7627-47c3-8786-0f48870ff5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:21.669648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-685ac80c-7627-47c3-8786-0f48870ff5ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:21.750944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:21.779096 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ac80c-7627-47c3-8786-0f48870ff5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:21.780525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ac80c-7627-47c3-8786-0f48870ff5ca service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:21.797901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 666/1335] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:24.298268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f442-5930-403d-be8d-cb5d728e70ef None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:24.298849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f442-5930-403d-be8d-cb5d728e70ef None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:24.298849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f442-5930-403d-be8d-cb5d728e70ef None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:24.300208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 670/1336] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:24.305841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3e2ce-e241-434c-9508-f2baee0b4c29 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:24.306261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3e2ce-e241-434c-9508-f2baee0b4c29 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:24.306432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3e2ce-e241-434c-9508-f2baee0b4c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:24.369319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-95c3e2ce-e241-434c-9508-f2baee0b4c29 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:24.370538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-95c3e2ce-e241-434c-9508-f2baee0b4c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:24.466382 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 667/1337] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:24.501259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45f442-5930-403d-be8d-cb5d728e70ef None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:24.502806 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-fd45f442-5930-403d-be8d-cb5d728e70ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:24.603640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45f442-5930-403d-be8d-cb5d728e70ef None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:24.675414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9aceee-1003-46ea-b739-5ea9af694f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:24.676621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9aceee-1003-46ea-b739-5ea9af694f8d service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:24.676903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9aceee-1003-46ea-b739-5ea9af694f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:24.677180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9aceee-1003-46ea-b739-5ea9af694f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:24.678837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9c9aceee-1003-46ea-b739-5ea9af694f8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:24.714534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:24.786543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9aceee-1003-46ea-b739-5ea9af694f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:24.787980 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9aceee-1003-46ea-b739-5ea9af694f8d service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:24.805993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 671/1338] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:28.902062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee57a7e-1e3d-4e78-bab4-707d403ac9a0 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:28.902707 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee57a7e-1e3d-4e78-bab4-707d403ac9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:28.902707 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee57a7e-1e3d-4e78-bab4-707d403ac9a0 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:28.904064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 668/1339] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:28.909308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1172eb7-716e-4fd3-8ade-a7c0bbe672c0 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:28.909719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1172eb7-716e-4fd3-8ade-a7c0bbe672c0 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:28.910585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1172eb7-716e-4fd3-8ade-a7c0bbe672c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:28.979816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-d1172eb7-716e-4fd3-8ade-a7c0bbe672c0 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:28.980991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d1172eb7-716e-4fd3-8ade-a7c0bbe672c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:29.071438 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:29.085044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 672/1340] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:29.122705 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee57a7e-1e3d-4e78-bab4-707d403ac9a0 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:29.124900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-cee57a7e-1e3d-4e78-bab4-707d403ac9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:29.178772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:29.226143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee57a7e-1e3d-4e78-bab4-707d403ac9a0 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:29.229692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c8dd2-93dc-45b4-b469-e29db4663685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:29.230366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8dd2-93dc-45b4-b469-e29db4663685 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:29.230658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8dd2-93dc-45b4-b469-e29db4663685 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:29.230938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8dd2-93dc-45b4-b469-e29db4663685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:29.239994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c8dd2-93dc-45b4-b469-e29db4663685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:29.241412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c8dd2-93dc-45b4-b469-e29db4663685 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:29.250827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 669/1341] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:32.074710 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d0c73-a34c-4134-b133-842e3f227186 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:32.075265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d0c73-a34c-4134-b133-842e3f227186 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:32.075265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d0c73-a34c-4134-b133-842e3f227186 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:32.076638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 673/1342] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:32.082671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff7c3f-e04c-492a-bfe9-56c793fd08ac None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:32.083080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff7c3f-e04c-492a-bfe9-56c793fd08ac None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:32.083385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff7c3f-e04c-492a-bfe9-56c793fd08ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:32.110115 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-0eff7c3f-e04c-492a-bfe9-56c793fd08ac None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:32.111314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-0eff7c3f-e04c-492a-bfe9-56c793fd08ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:32.185283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 670/1343] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:32.242164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfea5c-6772-4f63-bdf7-2550a6ba4497 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:32.242482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfea5c-6772-4f63-bdf7-2550a6ba4497 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:32.242760 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfea5c-6772-4f63-bdf7-2550a6ba4497 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:32.244144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 674/1344] 10.222.0.45 () {54 vars in 964 bytes} [Tue May 24 12:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:32.250006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a51243-7f28-4fae-a95f-17cbe0ae69f5 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:32.250405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a51243-7f28-4fae-a95f-17cbe0ae69f5 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:32.250783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a51243-7f28-4fae-a95f-17cbe0ae69f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:32.307963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-32a51243-7f28-4fae-a95f-17cbe0ae69f5 None None] MFA Rules not processed for user `bb111be6bc2946d3b50b8d99a89cf8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:32.309139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-32a51243-7f28-4fae-a95f-17cbe0ae69f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:32.315549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 671/1345] 10.222.0.45 () {58 vars in 1061 bytes} [Tue May 24 12:51:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:32.325126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dfea5c-6772-4f63-bdf7-2550a6ba4497 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:32.339028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dfea5c-6772-4f63-bdf7-2550a6ba4497 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:32.342531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4d5d5-091e-4b3f-8621-c9af402d39c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:32.343175 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d5d5-091e-4b3f-8621-c9af402d39c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:32.343455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d5d5-091e-4b3f-8621-c9af402d39c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:32.343726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d5d5-091e-4b3f-8621-c9af402d39c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:32.352959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4d5d5-091e-4b3f-8621-c9af402d39c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:32.354434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4d5d5-091e-4b3f-8621-c9af402d39c2 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:32.363676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 675/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:37.305480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a717a-81cc-41b9-93a1-0e96d6806473 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:37.306036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a717a-81cc-41b9-93a1-0e96d6806473 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:37.306036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a717a-81cc-41b9-93a1-0e96d6806473 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:37.307521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 672/1347] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:37.312923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b8a79-1c1d-45d0-9acc-4471404f9944 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:37.313343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b8a79-1c1d-45d0-9acc-4471404f9944 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:37.313618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b8a79-1c1d-45d0-9acc-4471404f9944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:37.332081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c52b8a79-1c1d-45d0-9acc-4471404f9944 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:37.333275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c52b8a79-1c1d-45d0-9acc-4471404f9944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:37.341337 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 676/1348] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:37.380017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a717a-81cc-41b9-93a1-0e96d6806473 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:37.392501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a717a-81cc-41b9-93a1-0e96d6806473 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:37.396424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842199af-61e5-473f-8ed3-b8cfc371c102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:37.397144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842199af-61e5-473f-8ed3-b8cfc371c102 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:37.397435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842199af-61e5-473f-8ed3-b8cfc371c102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:37.397717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842199af-61e5-473f-8ed3-b8cfc371c102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:37.407090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842199af-61e5-473f-8ed3-b8cfc371c102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:37.408522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842199af-61e5-473f-8ed3-b8cfc371c102 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:37.419056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 673/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:42.618670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f421f-1aab-4a2c-a268-6286d597dfb5 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:42.619608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f421f-1aab-4a2c-a268-6286d597dfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:42.620059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f421f-1aab-4a2c-a268-6286d597dfb5 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:42.621568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 677/1350] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 12:51:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:42.626585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93170066-e0f6-40ec-b5d5-a73496166df6 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:42.626983 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93170066-e0f6-40ec-b5d5-a73496166df6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:42.627141 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93170066-e0f6-40ec-b5d5-a73496166df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:42.682454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-93170066-e0f6-40ec-b5d5-a73496166df6 None None] MFA Rules not processed for user `630f0527bd844de9be072c459dab96c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:42.683887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-93170066-e0f6-40ec-b5d5-a73496166df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:42.697346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:42.823734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 674/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 12:51:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:47.017577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07faa644-1825-4970-a3e1-32889ad5325a None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:47.018106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07faa644-1825-4970-a3e1-32889ad5325a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:47.018106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07faa644-1825-4970-a3e1-32889ad5325a None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:47.020104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 678/1352] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 12:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:47.024965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5043e1-1fb6-4d89-b344-85432c32deb7 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:47.025749 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5043e1-1fb6-4d89-b344-85432c32deb7 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:47.026098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5043e1-1fb6-4d89-b344-85432c32deb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:47.050002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-2b5043e1-1fb6-4d89-b344-85432c32deb7 None None] MFA Rules not processed for user `630f0527bd844de9be072c459dab96c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:47.052578 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2b5043e1-1fb6-4d89-b344-85432c32deb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:47.062030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 675/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 12:51:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:47.326201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=13 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:51:48.621722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177a612-22de-4be3-bdb6-7d3d6a84964e None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:48.622359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177a612-22de-4be3-bdb6-7d3d6a84964e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:48.622359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177a612-22de-4be3-bdb6-7d3d6a84964e None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:48.623665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 679/1354] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 12:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:48.629251 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ae91-562b-43dc-a245-655ba6fb79f2 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:48.629678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ae91-562b-43dc-a245-655ba6fb79f2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:48.629810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ae91-562b-43dc-a245-655ba6fb79f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:48.648124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-f903ae91-562b-43dc-a245-655ba6fb79f2 None None] MFA Rules not processed for user `630f0527bd844de9be072c459dab96c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:48.649290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f903ae91-562b-43dc-a245-655ba6fb79f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:48.658102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 676/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 12:51:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:52.553955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40792e3c-1899-4b3e-af2a-251019e39b6d None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:52.554508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40792e3c-1899-4b3e-af2a-251019e39b6d None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:52.554508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40792e3c-1899-4b3e-af2a-251019e39b6d None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:52.555842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 680/1356] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:52.560978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f7452-a3cb-4aa1-9313-dbb9eeef56f1 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:52.561377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f7452-a3cb-4aa1-9313-dbb9eeef56f1 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:52.561534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f7452-a3cb-4aa1-9313-dbb9eeef56f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:52.579660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a56f7452-a3cb-4aa1-9313-dbb9eeef56f1 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:52.580845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a56f7452-a3cb-4aa1-9313-dbb9eeef56f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:52.588871 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 677/1357] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:52.643128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f396c-c932-4a39-8a2c-b30a849efa8d None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:52.643766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f396c-c932-4a39-8a2c-b30a849efa8d None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:52.644786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f396c-c932-4a39-8a2c-b30a849efa8d None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:52.646593 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 681/1358] 10.222.0.45 () {54 vars in 961 bytes} [Tue May 24 12:51:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:52.659232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74eed21-4f3b-4eba-88f0-fc0a01fa7819 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:52.660112 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74eed21-4f3b-4eba-88f0-fc0a01fa7819 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:52.660835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74eed21-4f3b-4eba-88f0-fc0a01fa7819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:52.729742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-b74eed21-4f3b-4eba-88f0-fc0a01fa7819 None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:52.731348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b74eed21-4f3b-4eba-88f0-fc0a01fa7819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:52.819585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 678/1359] 10.222.0.45 () {58 vars in 1058 bytes} [Tue May 24 12:51:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 12:51:52.827180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f396c-c932-4a39-8a2c-b30a849efa8d None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:52.844527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f396c-c932-4a39-8a2c-b30a849efa8d None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:52.850577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d262b-fe02-4571-b149-f91487743584 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:52.851400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d262b-fe02-4571-b149-f91487743584 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:52.851939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d262b-fe02-4571-b149-f91487743584 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:52.852372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d262b-fe02-4571-b149-f91487743584 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:52.862029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d262b-fe02-4571-b149-f91487743584 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:52.864461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d262b-fe02-4571-b149-f91487743584 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:52.875151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 682/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:55.302031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e249b3-ba83-43e7-872d-d6a861cbb4b8 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:55.302602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e249b3-ba83-43e7-872d-d6a861cbb4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:55.302602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e249b3-ba83-43e7-872d-d6a861cbb4b8 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:55.303979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 679/1361] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:55.309477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d869c-8455-4faa-b984-390c9838f097 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:55.309891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d869c-8455-4faa-b984-390c9838f097 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:55.310160 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d869c-8455-4faa-b984-390c9838f097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:55.329119 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-352d869c-8455-4faa-b984-390c9838f097 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:55.330509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-352d869c-8455-4faa-b984-390c9838f097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:55.339647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 683/1362] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:55.384317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e249b3-ba83-43e7-872d-d6a861cbb4b8 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:55.400112 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e249b3-ba83-43e7-872d-d6a861cbb4b8 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:55.409361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf28bc-9c39-400e-be5f-dba3f25b7386 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:55.410142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf28bc-9c39-400e-be5f-dba3f25b7386 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:55.410514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf28bc-9c39-400e-be5f-dba3f25b7386 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:55.410861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf28bc-9c39-400e-be5f-dba3f25b7386 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:55.419924 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf28bc-9c39-400e-be5f-dba3f25b7386 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:55.421568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf28bc-9c39-400e-be5f-dba3f25b7386 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:55.431625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 680/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:51:57.951519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a79f0-a31c-4358-b444-413f4f428cf3 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:57.952054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a79f0-a31c-4358-b444-413f4f428cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:57.952151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a79f0-a31c-4358-b444-413f4f428cf3 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:57.953650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 684/1364] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:57.958786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff22057-e06c-4942-a0f0-89865abb057d None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:57.959179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff22057-e06c-4942-a0f0-89865abb057d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:57.959352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff22057-e06c-4942-a0f0-89865abb057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:57.977568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-0ff22057-e06c-4942-a0f0-89865abb057d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:57.979002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-0ff22057-e06c-4942-a0f0-89865abb057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:57.987215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 681/1365] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:58.032414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1a586-235c-4715-9938-d2bb6b82f82f None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:58.032700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1a586-235c-4715-9938-d2bb6b82f82f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:58.032975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1a586-235c-4715-9938-d2bb6b82f82f None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:58.034489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 685/1366] 10.222.0.45 () {54 vars in 961 bytes} [Tue May 24 12:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:51:58.041209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11088c-276b-4e67-954f-46ad8f36e3e5 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:58.041503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11088c-276b-4e67-954f-46ad8f36e3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:58.041783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11088c-276b-4e67-954f-46ad8f36e3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:58.070860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7a11088c-276b-4e67-954f-46ad8f36e3e5 None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:51:58.072224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7a11088c-276b-4e67-954f-46ad8f36e3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:51:58.083837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 682/1367] 10.222.0.45 () {58 vars in 1058 bytes} [Tue May 24 12:51:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:51:58.091402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1a586-235c-4715-9938-d2bb6b82f82f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:51:58.105354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1a586-235c-4715-9938-d2bb6b82f82f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:51:58.110712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a80c7-2b64-4767-a7fb-59780e207605 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:51:58.111354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a80c7-2b64-4767-a7fb-59780e207605 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:51:58.111639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a80c7-2b64-4767-a7fb-59780e207605 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:51:58.111912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a80c7-2b64-4767-a7fb-59780e207605 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:51:58.120585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a80c7-2b64-4767-a7fb-59780e207605 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:51:58.122122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a80c7-2b64-4767-a7fb-59780e207605 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:51:58.131916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 686/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:00.391702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80920f-f5f3-46e7-8b1e-5d9d85a68bc9 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:00.392295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80920f-f5f3-46e7-8b1e-5d9d85a68bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:00.392295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80920f-f5f3-46e7-8b1e-5d9d85a68bc9 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:00.393631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 683/1369] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:00.399053 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda1ee8-7ce8-43ea-b1ba-317aa0a5edbc None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:00.399537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda1ee8-7ce8-43ea-b1ba-317aa0a5edbc None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:00.399627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda1ee8-7ce8-43ea-b1ba-317aa0a5edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:00.417606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8cda1ee8-7ce8-43ea-b1ba-317aa0a5edbc None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:00.418857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8cda1ee8-7ce8-43ea-b1ba-317aa0a5edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:00.427524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 687/1370] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:00.470762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80920f-f5f3-46e7-8b1e-5d9d85a68bc9 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:00.480087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:52:00.486057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80920f-f5f3-46e7-8b1e-5d9d85a68bc9 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:00.492292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62782c4-a058-47c2-822c-de7da087f3bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:00.493060 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62782c4-a058-47c2-822c-de7da087f3bc service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:00.493449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62782c4-a058-47c2-822c-de7da087f3bc service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:00.493959 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62782c4-a058-47c2-822c-de7da087f3bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:00.502764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62782c4-a058-47c2-822c-de7da087f3bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:00.504980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62782c4-a058-47c2-822c-de7da087f3bc service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:00.514795 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 684/1371] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:00.858021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f592a64-db62-492f-bdc5-74deb268fb78 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:00.858489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f592a64-db62-492f-bdc5-74deb268fb78 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:00.858857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f592a64-db62-492f-bdc5-74deb268fb78 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:00.860307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 688/1372] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:00.866225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b25ce5-e6c1-477f-9701-7041dab19a3b None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:00.866548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b25ce5-e6c1-477f-9701-7041dab19a3b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:00.867009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b25ce5-e6c1-477f-9701-7041dab19a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:00.886215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a7b25ce5-e6c1-477f-9701-7041dab19a3b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:00.887762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a7b25ce5-e6c1-477f-9701-7041dab19a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:00.896530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 685/1373] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:01.202592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b7d59-366b-4871-ac0f-c8383538ace3 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:01.203034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b7d59-366b-4871-ac0f-c8383538ace3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:01.203301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b7d59-366b-4871-ac0f-c8383538ace3 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:01.204896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 689/1374] 10.222.0.45 () {54 vars in 963 bytes} [Tue May 24 12:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:01.211450 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f45087-3e42-4b94-b70e-3c0190c79d9f None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:01.211826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f45087-3e42-4b94-b70e-3c0190c79d9f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:01.212178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f45087-3e42-4b94-b70e-3c0190c79d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:01.262447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-67f45087-3e42-4b94-b70e-3c0190c79d9f None None] MFA Rules not processed for user `8c9f08515d514be8af1176717ec9e6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:01.263746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-67f45087-3e42-4b94-b70e-3c0190c79d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:01.325967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 686/1375] 10.222.0.45 () {58 vars in 1060 bytes} [Tue May 24 12:52:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:01.334400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b7d59-366b-4871-ac0f-c8383538ace3 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:01.345369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b7d59-366b-4871-ac0f-c8383538ace3 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:01.349732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9471cb-b92e-4d2f-a3c0-b3bbb976f5a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:01.350511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9471cb-b92e-4d2f-a3c0-b3bbb976f5a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:01.350865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9471cb-b92e-4d2f-a3c0-b3bbb976f5a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:01.351212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9471cb-b92e-4d2f-a3c0-b3bbb976f5a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:01.360929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9471cb-b92e-4d2f-a3c0-b3bbb976f5a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:01.362492 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9471cb-b92e-4d2f-a3c0-b3bbb976f5a6 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:01.372629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 690/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:02.778464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151482-b3d7-4d61-b421-e0036fc9fb4c None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:02.779042 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151482-b3d7-4d61-b421-e0036fc9fb4c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:02.779042 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151482-b3d7-4d61-b421-e0036fc9fb4c None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:02.780403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 687/1377] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:02.786067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9bd47-7f0f-4aca-8936-5b1c91df783f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:02.786558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9bd47-7f0f-4aca-8936-5b1c91df783f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:02.786713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9bd47-7f0f-4aca-8936-5b1c91df783f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:02.804726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-73a9bd47-7f0f-4aca-8936-5b1c91df783f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:02.805928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-73a9bd47-7f0f-4aca-8936-5b1c91df783f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:02.814451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 691/1378] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:02.860687 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c151482-b3d7-4d61-b421-e0036fc9fb4c None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:02.877094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c151482-b3d7-4d61-b421-e0036fc9fb4c None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:02.883267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f0afe-8958-48fb-8ede-bdd1b9eba33f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:02.883953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f0afe-8958-48fb-8ede-bdd1b9eba33f service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:02.884243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f0afe-8958-48fb-8ede-bdd1b9eba33f service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:02.884527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f0afe-8958-48fb-8ede-bdd1b9eba33f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:02.893777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f0afe-8958-48fb-8ede-bdd1b9eba33f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:02.895496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f0afe-8958-48fb-8ede-bdd1b9eba33f service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:02.906104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 688/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:05.178482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05804175-61bf-4151-a589-f5468cb70304 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:05.179007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05804175-61bf-4151-a589-f5468cb70304 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:05.179007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05804175-61bf-4151-a589-f5468cb70304 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:05.180569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 692/1380] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:05.185865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79a7e4-2753-44f9-8295-8e9c63b1e159 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:05.186417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79a7e4-2753-44f9-8295-8e9c63b1e159 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:05.186526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79a7e4-2753-44f9-8295-8e9c63b1e159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:05.206503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5e79a7e4-2753-44f9-8295-8e9c63b1e159 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:05.207868 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5e79a7e4-2753-44f9-8295-8e9c63b1e159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:05.224006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 689/1381] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:05.273112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05804175-61bf-4151-a589-f5468cb70304 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:05.287724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05804175-61bf-4151-a589-f5468cb70304 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:05.294248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a87d8-3d44-489d-a2ea-bc5e9f5c3c00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:05.295030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a87d8-3d44-489d-a2ea-bc5e9f5c3c00 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:05.295408 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a87d8-3d44-489d-a2ea-bc5e9f5c3c00 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:05.295792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a87d8-3d44-489d-a2ea-bc5e9f5c3c00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:05.304602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a87d8-3d44-489d-a2ea-bc5e9f5c3c00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:05.306210 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a87d8-3d44-489d-a2ea-bc5e9f5c3c00 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:05.316950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 693/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:07.545398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1c5b7-44f0-41e5-bd39-626ce08b885f None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:07.545916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1c5b7-44f0-41e5-bd39-626ce08b885f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:07.546047 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1c5b7-44f0-41e5-bd39-626ce08b885f None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:07.547361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 690/1383] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:07.552775 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cd46a-9ea2-463a-81bc-1c860ba59857 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:07.553203 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cd46a-9ea2-463a-81bc-1c860ba59857 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:07.553385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cd46a-9ea2-463a-81bc-1c860ba59857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:07.572259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-d32cd46a-9ea2-463a-81bc-1c860ba59857 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:07.573444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d32cd46a-9ea2-463a-81bc-1c860ba59857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:07.584122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 694/1384] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:07.624097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1c5b7-44f0-41e5-bd39-626ce08b885f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:07.639953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1c5b7-44f0-41e5-bd39-626ce08b885f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:07.646358 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f7ef1-613a-48b9-8ada-5b81b88a6714 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:07.647197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f7ef1-613a-48b9-8ada-5b81b88a6714 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:07.647588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f7ef1-613a-48b9-8ada-5b81b88a6714 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:07.648083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f7ef1-613a-48b9-8ada-5b81b88a6714 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:07.657259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f7ef1-613a-48b9-8ada-5b81b88a6714 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:07.659053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f7ef1-613a-48b9-8ada-5b81b88a6714 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:07.669481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 691/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:09.965515 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4049f-59bb-4a73-9c6b-f256bbfdc9cc None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:09.966117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4049f-59bb-4a73-9c6b-f256bbfdc9cc None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:09.966117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4049f-59bb-4a73-9c6b-f256bbfdc9cc None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:09.967491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 695/1386] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:09.973143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7baba0-07a7-4076-811b-356ce5006eaf None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:09.973678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7baba0-07a7-4076-811b-356ce5006eaf None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:09.973678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7baba0-07a7-4076-811b-356ce5006eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:09.991954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1f7baba0-07a7-4076-811b-356ce5006eaf None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:09.993310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1f7baba0-07a7-4076-811b-356ce5006eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:10.001259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 692/1387] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:10.045717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4049f-59bb-4a73-9c6b-f256bbfdc9cc None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:10.060663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4049f-59bb-4a73-9c6b-f256bbfdc9cc None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:10.066668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b2989-d764-4a5c-94a8-9125c2715033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:10.067303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2989-d764-4a5c-94a8-9125c2715033 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:10.067531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2989-d764-4a5c-94a8-9125c2715033 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:10.067805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2989-d764-4a5c-94a8-9125c2715033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:10.076080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b2989-d764-4a5c-94a8-9125c2715033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:10.077669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b2989-d764-4a5c-94a8-9125c2715033 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:10.087100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 696/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:12.333663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c615ba0-79f9-43f8-9467-4ef277b2a5e8 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:12.334178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c615ba0-79f9-43f8-9467-4ef277b2a5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:12.334465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c615ba0-79f9-43f8-9467-4ef277b2a5e8 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:12.336358 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 693/1389] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:52:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:12.341834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90639ab7-3dfc-4292-8c99-f9f24c5ad83e None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:12.342321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90639ab7-3dfc-4292-8c99-f9f24c5ad83e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:12.342422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90639ab7-3dfc-4292-8c99-f9f24c5ad83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:12.360619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-90639ab7-3dfc-4292-8c99-f9f24c5ad83e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:12.361825 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-90639ab7-3dfc-4292-8c99-f9f24c5ad83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:12.369951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 697/1390] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:12.415144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c615ba0-79f9-43f8-9467-4ef277b2a5e8 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:12.433150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c615ba0-79f9-43f8-9467-4ef277b2a5e8 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:12.439949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447d972-13a7-492f-ac1a-49d236fd38e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:12.440788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447d972-13a7-492f-ac1a-49d236fd38e5 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:12.441135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447d972-13a7-492f-ac1a-49d236fd38e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:12.441521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447d972-13a7-492f-ac1a-49d236fd38e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:12.452148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447d972-13a7-492f-ac1a-49d236fd38e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:12.454149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447d972-13a7-492f-ac1a-49d236fd38e5 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:12.467450 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 694/1391] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:14.740293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa47d8-d30b-4bc1-a2d2-a180164a1036 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:14.740872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa47d8-d30b-4bc1-a2d2-a180164a1036 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:14.740872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa47d8-d30b-4bc1-a2d2-a180164a1036 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:14.742230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 698/1392] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:14.748945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a6a7a-1d3d-46ba-b5de-9d8db02a9887 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:14.749388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a6a7a-1d3d-46ba-b5de-9d8db02a9887 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:14.749721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a6a7a-1d3d-46ba-b5de-9d8db02a9887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:14.771309 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-4b2a6a7a-1d3d-46ba-b5de-9d8db02a9887 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:14.772793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4b2a6a7a-1d3d-46ba-b5de-9d8db02a9887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:14.782475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 695/1393] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:14.832480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa47d8-d30b-4bc1-a2d2-a180164a1036 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:14.846765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa47d8-d30b-4bc1-a2d2-a180164a1036 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:14.852966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111360c-193c-4ba9-af69-1a86a9ed5af9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:14.853643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111360c-193c-4ba9-af69-1a86a9ed5af9 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:14.853935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111360c-193c-4ba9-af69-1a86a9ed5af9 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:14.854215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111360c-193c-4ba9-af69-1a86a9ed5af9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:14.863381 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111360c-193c-4ba9-af69-1a86a9ed5af9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:14.865029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111360c-193c-4ba9-af69-1a86a9ed5af9 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:14.874863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 699/1394] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 12:52:17.104781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ae7d-db08-4013-aa42-ac7121cc7430 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:17.105333 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ae7d-db08-4013-aa42-ac7121cc7430 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:17.105521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ae7d-db08-4013-aa42-ac7121cc7430 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:17.107134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 696/1395] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:52:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:17.112899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb56fe2-f365-434c-a93a-fcd7c7c87bc2 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:17.112899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb56fe2-f365-434c-a93a-fcd7c7c87bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:17.113443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb56fe2-f365-434c-a93a-fcd7c7c87bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:17.134102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6cb56fe2-f365-434c-a93a-fcd7c7c87bc2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:17.135356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6cb56fe2-f365-434c-a93a-fcd7c7c87bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:17.143193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 700/1396] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:17.187662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ae7d-db08-4013-aa42-ac7121cc7430 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:17.205631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ae7d-db08-4013-aa42-ac7121cc7430 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:17.212929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d987e-d9fe-41cd-b3ea-b38a517f5a26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:17.213578 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d987e-d9fe-41cd-b3ea-b38a517f5a26 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:17.213859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d987e-d9fe-41cd-b3ea-b38a517f5a26 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:17.214133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d987e-d9fe-41cd-b3ea-b38a517f5a26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:17.223782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d987e-d9fe-41cd-b3ea-b38a517f5a26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:17.225825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d987e-d9fe-41cd-b3ea-b38a517f5a26 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:17.236654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 697/1397] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:19.484486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e4c5e-519b-4e5b-8685-787d8cd4e2ab None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:19.485081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e4c5e-519b-4e5b-8685-787d8cd4e2ab None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:19.485081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e4c5e-519b-4e5b-8685-787d8cd4e2ab None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:19.486445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 701/1398] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:19.492340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3578ff-a942-4e6f-bb14-6a4d919fecbc None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:19.492772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3578ff-a942-4e6f-bb14-6a4d919fecbc None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:19.492872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3578ff-a942-4e6f-bb14-6a4d919fecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:19.511612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9d3578ff-a942-4e6f-bb14-6a4d919fecbc None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:19.512903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9d3578ff-a942-4e6f-bb14-6a4d919fecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:19.521124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 698/1399] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:19.567295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e4c5e-519b-4e5b-8685-787d8cd4e2ab None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:19.575877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=55 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:52:19.582696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e4c5e-519b-4e5b-8685-787d8cd4e2ab None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:19.588156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af882d-d729-4c4d-a0f7-8ffea3fcc5c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:19.588804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af882d-d729-4c4d-a0f7-8ffea3fcc5c5 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:19.589094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af882d-d729-4c4d-a0f7-8ffea3fcc5c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:19.589376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af882d-d729-4c4d-a0f7-8ffea3fcc5c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:19.598764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af882d-d729-4c4d-a0f7-8ffea3fcc5c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:19.600365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af882d-d729-4c4d-a0f7-8ffea3fcc5c5 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:19.610083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 702/1400] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:19.726308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323e736-07ad-4f1d-b820-f359ea892b24 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:19.726610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323e736-07ad-4f1d-b820-f359ea892b24 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:19.726896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323e736-07ad-4f1d-b820-f359ea892b24 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:19.728334 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 699/1401] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:19.734485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f632728-b8ad-4319-9a7a-0c4c19df1e21 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:19.734777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f632728-b8ad-4319-9a7a-0c4c19df1e21 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:19.735056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f632728-b8ad-4319-9a7a-0c4c19df1e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:19.754073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-2f632728-b8ad-4319-9a7a-0c4c19df1e21 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:19.755340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2f632728-b8ad-4319-9a7a-0c4c19df1e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:19.763487 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 703/1402] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:19.821106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750d8f-b220-4330-99dc-ae80bab5c4be None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:19.821396 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750d8f-b220-4330-99dc-ae80bab5c4be None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:19.821672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750d8f-b220-4330-99dc-ae80bab5c4be None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:19.823155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 700/1403] 10.222.0.45 () {54 vars in 962 bytes} [Tue May 24 12:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:19.829290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011baac8-9a0c-4a7d-b241-7fc45f0e2dda None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:19.829661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011baac8-9a0c-4a7d-b241-7fc45f0e2dda None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:19.830000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011baac8-9a0c-4a7d-b241-7fc45f0e2dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:19.888388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-011baac8-9a0c-4a7d-b241-7fc45f0e2dda None None] MFA Rules not processed for user `e47f1d65687e4cfcbce764076141702c`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:19.889911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-011baac8-9a0c-4a7d-b241-7fc45f0e2dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:19.949367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 704/1404] 10.222.0.45 () {58 vars in 1059 bytes} [Tue May 24 12:52:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 12:52:19.957711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56750d8f-b220-4330-99dc-ae80bab5c4be None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:19.968166 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56750d8f-b220-4330-99dc-ae80bab5c4be None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:19.971624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531d541-88bc-47e3-914c-6dba0fb17baa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e47f1d65687e4cfcbce764076141702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:19.972809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d541-88bc-47e3-914c-6dba0fb17baa service glance] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:19.973089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d541-88bc-47e3-914c-6dba0fb17baa service glance] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:19.973360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d541-88bc-47e3-914c-6dba0fb17baa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:19.982058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531d541-88bc-47e3-914c-6dba0fb17baa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:19.983622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531d541-88bc-47e3-914c-6dba0fb17baa service glance] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:19.992776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 701/1405] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 12:52:20.514425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628d7e1-b2c1-4586-914e-07876d5b1026 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:20.514922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628d7e1-b2c1-4586-914e-07876d5b1026 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:20.515018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628d7e1-b2c1-4586-914e-07876d5b1026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:20.561911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c628d7e1-b2c1-4586-914e-07876d5b1026 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:20.563118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c628d7e1-b2c1-4586-914e-07876d5b1026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:20.623927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 705/1406] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 12:52:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:20.666906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379a447-e29b-4bcd-9210-a854a3457de4 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:20.667363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379a447-e29b-4bcd-9210-a854a3457de4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:20.667462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379a447-e29b-4bcd-9210-a854a3457de4 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:20.668818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 702/1407] 10.222.0.45 () {54 vars in 942 bytes} [Tue May 24 12:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:20.675399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb1b5d-56ae-4bf9-b323-0034c642ce52 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:20.675693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb1b5d-56ae-4bf9-b323-0034c642ce52 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:20.675965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb1b5d-56ae-4bf9-b323-0034c642ce52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:20.720606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:52:20.723626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b5fb1b5d-56ae-4bf9-b323-0034c642ce52 None None] MFA Rules not processed for user `d00b728319304b1e9ba2f1ee93022187`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:20.725228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b5fb1b5d-56ae-4bf9-b323-0034c642ce52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:20.795659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 706/1408] 10.222.0.45 () {58 vars in 1039 bytes} [Tue May 24 12:52:20 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:20.805109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379a447-e29b-4bcd-9210-a854a3457de4 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:20.818721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379a447-e29b-4bcd-9210-a854a3457de4 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:20.823577 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d9a3d-f9a3-46ab-9338-6cd5783abe9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:20.824211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d9a3d-f9a3-46ab-9338-6cd5783abe9c service swift] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:20.824498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d9a3d-f9a3-46ab-9338-6cd5783abe9c service swift] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:20.824778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d9a3d-f9a3-46ab-9338-6cd5783abe9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:20.833125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d9a3d-f9a3-46ab-9338-6cd5783abe9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:20.834754 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d9a3d-f9a3-46ab-9338-6cd5783abe9c service swift] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:20.842557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 703/1409] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 12:52:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 12:52:30.792668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=30 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:52:30.839428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=93 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:52:44.364335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43a79-2029-4f5d-8c7b-c38ff4d9a75f None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:44.364973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43a79-2029-4f5d-8c7b-c38ff4d9a75f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:44.364973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43a79-2029-4f5d-8c7b-c38ff4d9a75f None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:44.366539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 707/1410] 10.222.0.45 () {54 vars in 918 bytes} [Tue May 24 12:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:44.371328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8021a-c64e-43a8-9933-0ddc5bdca387 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:44.371812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8021a-c64e-43a8-9933-0ddc5bdca387 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:44.371909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8021a-c64e-43a8-9933-0ddc5bdca387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:44.390852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a8f8021a-c64e-43a8-9933-0ddc5bdca387 None None] MFA Rules not processed for user `630f0527bd844de9be072c459dab96c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:44.392075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a8f8021a-c64e-43a8-9933-0ddc5bdca387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:44.401270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 704/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Tue May 24 12:52:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:44.415254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce2c8a-2a07-41ee-8217-482640b6dce4 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:44.415546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce2c8a-2a07-41ee-8217-482640b6dce4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:44.415831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce2c8a-2a07-41ee-8217-482640b6dce4 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:44.417195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 708/1412] 10.222.0.45 () {54 vars in 918 bytes} [Tue May 24 12:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:44.421600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c71f60-d051-4b06-9870-edb9408aef16 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:44.421894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c71f60-d051-4b06-9870-edb9408aef16 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:44.422204 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c71f60-d051-4b06-9870-edb9408aef16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:44.440262 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e8c71f60-d051-4b06-9870-edb9408aef16 None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:44.442004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e8c71f60-d051-4b06-9870-edb9408aef16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:44.451094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 705/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Tue May 24 12:52:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:47.083449 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940e32-271f-4991-b8f1-9ddf942093ac None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:47.083983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940e32-271f-4991-b8f1-9ddf942093ac None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:47.083983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940e32-271f-4991-b8f1-9ddf942093ac None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:47.085402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:47.090724 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b535216-d7c5-45e1-a0a9-abd9c74ad7d2 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:47.091133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b535216-d7c5-45e1-a0a9-abd9c74ad7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:47.091277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b535216-d7c5-45e1-a0a9-abd9c74ad7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:47.108949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-2b535216-d7c5-45e1-a0a9-abd9c74ad7d2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:47.110119 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2b535216-d7c5-45e1-a0a9-abd9c74ad7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:47.118363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:47.160589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39940e32-271f-4991-b8f1-9ddf942093ac None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:47.175404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39940e32-271f-4991-b8f1-9ddf942093ac None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:47.180847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be8a39-f9d7-44e4-9d0c-e07c7978b9c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:47.181515 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8a39-f9d7-44e4-9d0c-e07c7978b9c4 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:47.181800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8a39-f9d7-44e4-9d0c-e07c7978b9c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:47.182078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8a39-f9d7-44e4-9d0c-e07c7978b9c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:47.190173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be8a39-f9d7-44e4-9d0c-e07c7978b9c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:47.192797 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be8a39-f9d7-44e4-9d0c-e07c7978b9c4 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:47.202151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 710/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:48.154209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f13ef-a381-4d08-934c-47ad0b401028 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:48.154810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f13ef-a381-4d08-934c-47ad0b401028 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:48.154810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f13ef-a381-4d08-934c-47ad0b401028 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:48.156161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:48.161165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8df0-4ed8-4c0c-8515-a59259e94d4b None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:48.161591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8df0-4ed8-4c0c-8515-a59259e94d4b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:48.161726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8df0-4ed8-4c0c-8515-a59259e94d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:48.181004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-235d8df0-4ed8-4c0c-8515-a59259e94d4b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:48.182181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-235d8df0-4ed8-4c0c-8515-a59259e94d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:48.189863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:48.352787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f13ef-a381-4d08-934c-47ad0b401028 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:48.367274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f13ef-a381-4d08-934c-47ad0b401028 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:48.372564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab9200-7e49-442b-ab3b-ae983afaeb25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:48.373205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab9200-7e49-442b-ab3b-ae983afaeb25 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:48.373491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab9200-7e49-442b-ab3b-ae983afaeb25 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:48.373770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab9200-7e49-442b-ab3b-ae983afaeb25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:48.384489 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab9200-7e49-442b-ab3b-ae983afaeb25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:48.384489 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab9200-7e49-442b-ab3b-ae983afaeb25 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:48.394918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 708/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:49.309347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c697c-40fa-4c27-b610-ee78492150b6 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:49.309929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c697c-40fa-4c27-b610-ee78492150b6 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:49.309929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c697c-40fa-4c27-b610-ee78492150b6 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:49.311299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:49.316643 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecaae5-ae37-4428-a334-7c09a80a306c None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:49.317061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecaae5-ae37-4428-a334-7c09a80a306c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:49.317200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecaae5-ae37-4428-a334-7c09a80a306c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:49.335253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-daecaae5-ae37-4428-a334-7c09a80a306c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:49.336460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-daecaae5-ae37-4428-a334-7c09a80a306c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:49.347933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:49.499203 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c697c-40fa-4c27-b610-ee78492150b6 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:49.514306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c697c-40fa-4c27-b610-ee78492150b6 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:49.520304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ff583-4d60-4fdb-add2-72fc80ca1cf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:49.520943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ff583-4d60-4fdb-add2-72fc80ca1cf0 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:49.521230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ff583-4d60-4fdb-add2-72fc80ca1cf0 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:49.521507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ff583-4d60-4fdb-add2-72fc80ca1cf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:49.530020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ff583-4d60-4fdb-add2-72fc80ca1cf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:49.531635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ff583-4d60-4fdb-add2-72fc80ca1cf0 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:49.540901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 713/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:50.434380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069bd6a-7fa6-4b00-8d12-2ca71736500f None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:50.434834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069bd6a-7fa6-4b00-8d12-2ca71736500f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:50.434929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069bd6a-7fa6-4b00-8d12-2ca71736500f None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:50.436302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:50.441802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753996da-80de-44ba-b1a0-2aefb4038d50 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:50.442208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753996da-80de-44ba-b1a0-2aefb4038d50 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:50.442377 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753996da-80de-44ba-b1a0-2aefb4038d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:50.459717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-753996da-80de-44ba-b1a0-2aefb4038d50 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:50.460919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-753996da-80de-44ba-b1a0-2aefb4038d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:50.468490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:50.614926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069bd6a-7fa6-4b00-8d12-2ca71736500f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:50.629655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069bd6a-7fa6-4b00-8d12-2ca71736500f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:50.634907 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9f60e-b60c-42c3-8c9f-52acdb0922e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:50.635532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9f60e-b60c-42c3-8c9f-52acdb0922e9 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:50.635815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9f60e-b60c-42c3-8c9f-52acdb0922e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:50.636088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9f60e-b60c-42c3-8c9f-52acdb0922e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:50.644025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9f60e-b60c-42c3-8c9f-52acdb0922e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:50.645591 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9f60e-b60c-42c3-8c9f-52acdb0922e9 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:50.654731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 711/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:51.555990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32eef0-7584-441f-8521-526c3258bb43 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:51.556536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32eef0-7584-441f-8521-526c3258bb43 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:51.556536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32eef0-7584-441f-8521-526c3258bb43 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:51.557915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:51.562814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e831d16-b2da-491d-af3b-776c2f64170e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:51.563277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e831d16-b2da-491d-af3b-776c2f64170e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:51.563378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e831d16-b2da-491d-af3b-776c2f64170e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:51.582688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-4e831d16-b2da-491d-af3b-776c2f64170e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:51.583872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4e831d16-b2da-491d-af3b-776c2f64170e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:51.591500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 12:52:51.739753 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32eef0-7584-441f-8521-526c3258bb43 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:51.754481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32eef0-7584-441f-8521-526c3258bb43 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:51.759751 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6e376-2269-4aff-b5c5-d74fc13f18a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:51.760379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6e376-2269-4aff-b5c5-d74fc13f18a3 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:51.760662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6e376-2269-4aff-b5c5-d74fc13f18a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:51.760938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6e376-2269-4aff-b5c5-d74fc13f18a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:51.769393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6e376-2269-4aff-b5c5-d74fc13f18a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:51.771001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6e376-2269-4aff-b5c5-d74fc13f18a3 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:51.780161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 716/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:52.616185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742525c-9b73-4b38-b4a7-61a9836d72b7 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:52.616715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742525c-9b73-4b38-b4a7-61a9836d72b7 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:52.616897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742525c-9b73-4b38-b4a7-61a9836d72b7 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:52.618310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:52.623275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdd974-781a-4e08-bc53-896549d22e49 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:52.623723 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdd974-781a-4e08-bc53-896549d22e49 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:52.623819 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdd974-781a-4e08-bc53-896549d22e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:52.642516 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-18bdd974-781a-4e08-bc53-896549d22e49 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:52.644333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-18bdd974-781a-4e08-bc53-896549d22e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:52.652154 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:52.802116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742525c-9b73-4b38-b4a7-61a9836d72b7 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:52.816685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742525c-9b73-4b38-b4a7-61a9836d72b7 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:52.821921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acffae2-3430-4132-bcee-9ba7ad40e14c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:52.822589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acffae2-3430-4132-bcee-9ba7ad40e14c service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:52.822873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acffae2-3430-4132-bcee-9ba7ad40e14c service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:52.823147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acffae2-3430-4132-bcee-9ba7ad40e14c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:52.830977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acffae2-3430-4132-bcee-9ba7ad40e14c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:52.832543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acffae2-3430-4132-bcee-9ba7ad40e14c service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:52.841638 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 714/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:53.683409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd468b4-1ee4-482a-b1dc-9c61f2008cd0 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:53.684035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd468b4-1ee4-482a-b1dc-9c61f2008cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:53.684035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd468b4-1ee4-482a-b1dc-9c61f2008cd0 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:53.685350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:53.691201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b37b11-fcca-4680-82e7-e2dfe1ae3fd1 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:53.691686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b37b11-fcca-4680-82e7-e2dfe1ae3fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:53.691788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b37b11-fcca-4680-82e7-e2dfe1ae3fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:53.711631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-25b37b11-fcca-4680-82e7-e2dfe1ae3fd1 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:53.712817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-25b37b11-fcca-4680-82e7-e2dfe1ae3fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:53.720435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:53.869452 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd468b4-1ee4-482a-b1dc-9c61f2008cd0 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:53.884063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd468b4-1ee4-482a-b1dc-9c61f2008cd0 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:53.889303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc3115-ee49-4e3f-ae49-0fbc8e6d44ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:53.889929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc3115-ee49-4e3f-ae49-0fbc8e6d44ba service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:53.890212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc3115-ee49-4e3f-ae49-0fbc8e6d44ba service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:53.890512 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc3115-ee49-4e3f-ae49-0fbc8e6d44ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:53.898463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc3115-ee49-4e3f-ae49-0fbc8e6d44ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:53.900044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc3115-ee49-4e3f-ae49-0fbc8e6d44ba service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:53.909774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 719/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:54.761470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3999a-56d4-495c-aaed-c9fe65569972 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:54.761966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3999a-56d4-495c-aaed-c9fe65569972 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:54.762088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3999a-56d4-495c-aaed-c9fe65569972 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:54.763437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:54.768350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d598d3-0fbb-46e5-90af-73be2dbac149 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:54.769293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d598d3-0fbb-46e5-90af-73be2dbac149 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:54.769400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d598d3-0fbb-46e5-90af-73be2dbac149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:54.786979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-40d598d3-0fbb-46e5-90af-73be2dbac149 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:54.788198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-40d598d3-0fbb-46e5-90af-73be2dbac149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:54.795919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:54.943442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3999a-56d4-495c-aaed-c9fe65569972 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:54.958270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3999a-56d4-495c-aaed-c9fe65569972 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:54.963568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba02ba-be89-4b18-bc22-d12fa29ea155 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:54.964206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba02ba-be89-4b18-bc22-d12fa29ea155 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:54.964492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba02ba-be89-4b18-bc22-d12fa29ea155 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:54.964774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba02ba-be89-4b18-bc22-d12fa29ea155 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:54.973371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba02ba-be89-4b18-bc22-d12fa29ea155 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:54.974990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba02ba-be89-4b18-bc22-d12fa29ea155 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:54.984108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 717/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:55.834075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4194a1b-d27b-491b-9dc0-e4ceeaf66a05 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:55.834641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4194a1b-d27b-491b-9dc0-e4ceeaf66a05 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:55.834641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4194a1b-d27b-491b-9dc0-e4ceeaf66a05 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:55.836044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:55.841069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad8de3-5095-4b57-8232-83c042a7a06e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:55.841454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad8de3-5095-4b57-8232-83c042a7a06e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:55.841633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad8de3-5095-4b57-8232-83c042a7a06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:55.859011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-bbad8de3-5095-4b57-8232-83c042a7a06e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:55.860185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-bbad8de3-5095-4b57-8232-83c042a7a06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:55.868270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:56.015215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4194a1b-d27b-491b-9dc0-e4ceeaf66a05 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:56.029556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4194a1b-d27b-491b-9dc0-e4ceeaf66a05 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:56.035497 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266d1a6-495f-4a15-86e0-5193e2f5e5d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:56.036136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266d1a6-495f-4a15-86e0-5193e2f5e5d2 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:56.036426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266d1a6-495f-4a15-86e0-5193e2f5e5d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:56.036706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266d1a6-495f-4a15-86e0-5193e2f5e5d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:56.044693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266d1a6-495f-4a15-86e0-5193e2f5e5d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:56.046352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266d1a6-495f-4a15-86e0-5193e2f5e5d2 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:56.056214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 722/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:52:58.139088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacca77-5dc5-485d-a3f1-3b04bfc0a307 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:58.139662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacca77-5dc5-485d-a3f1-3b04bfc0a307 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:58.139662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacca77-5dc5-485d-a3f1-3b04bfc0a307 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:58.141021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:52:58.146750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e42119-37d5-41dc-bf83-740c2b9004a6 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:58.147163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e42119-37d5-41dc-bf83-740c2b9004a6 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:58.147300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e42119-37d5-41dc-bf83-740c2b9004a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:58.165139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c3e42119-37d5-41dc-bf83-740c2b9004a6 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:52:58.166381 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c3e42119-37d5-41dc-bf83-740c2b9004a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:52:58.174165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:52:58.212013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eacca77-5dc5-485d-a3f1-3b04bfc0a307 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:52:58.223388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eacca77-5dc5-485d-a3f1-3b04bfc0a307 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:52:58.226928 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a87cde-ea01-42ef-a33a-188998aa1f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:52:58.227604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a87cde-ea01-42ef-a33a-188998aa1f73 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:52:58.227889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a87cde-ea01-42ef-a33a-188998aa1f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:52:58.228169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a87cde-ea01-42ef-a33a-188998aa1f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:52:58.237111 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a87cde-ea01-42ef-a33a-188998aa1f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:52:58.238600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a87cde-ea01-42ef-a33a-188998aa1f73 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:52:58.247800 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 720/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:00.643086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f38c03-6f04-4f15-b4b7-8de0f27d4fd2 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:00.643647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f38c03-6f04-4f15-b4b7-8de0f27d4fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:00.643647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f38c03-6f04-4f15-b4b7-8de0f27d4fd2 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:00.645035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:00.650253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0faf5-7bea-4d08-9dae-9b87c3d1cadd None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:00.650731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0faf5-7bea-4d08-9dae-9b87c3d1cadd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:00.650838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0faf5-7bea-4d08-9dae-9b87c3d1cadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:00.668712 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-35d0faf5-7bea-4d08-9dae-9b87c3d1cadd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:00.669829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-35d0faf5-7bea-4d08-9dae-9b87c3d1cadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:00.678124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:53:00.714722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f38c03-6f04-4f15-b4b7-8de0f27d4fd2 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:00.726241 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f38c03-6f04-4f15-b4b7-8de0f27d4fd2 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:00.729756 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298db2e0-8db4-421b-b092-624248c70530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:00.730418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298db2e0-8db4-421b-b092-624248c70530 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:00.730704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298db2e0-8db4-421b-b092-624248c70530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:00.730981 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298db2e0-8db4-421b-b092-624248c70530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:00.739890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298db2e0-8db4-421b-b092-624248c70530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:00.741336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298db2e0-8db4-421b-b092-624248c70530 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:00.750554 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 725/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:03.212435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33e324-0cfe-4969-aef4-4ec0298c0fab None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:03.213026 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33e324-0cfe-4969-aef4-4ec0298c0fab None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:03.213026 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33e324-0cfe-4969-aef4-4ec0298c0fab None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:03.214380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:03.219671 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965f0ff-d3b6-4a55-ad78-1deb705389e1 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:03.220075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965f0ff-d3b6-4a55-ad78-1deb705389e1 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:03.220231 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965f0ff-d3b6-4a55-ad78-1deb705389e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:03.238064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a965f0ff-d3b6-4a55-ad78-1deb705389e1 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:03.239326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a965f0ff-d3b6-4a55-ad78-1deb705389e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:03.247187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 12:53:03.285131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33e324-0cfe-4969-aef4-4ec0298c0fab None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:03.295590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33e324-0cfe-4969-aef4-4ec0298c0fab None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:03.299073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c5da0-a2ee-4dff-9715-1b222ca006bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:03.300217 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c5da0-a2ee-4dff-9715-1b222ca006bd service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:03.300499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c5da0-a2ee-4dff-9715-1b222ca006bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:03.300776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c5da0-a2ee-4dff-9715-1b222ca006bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:03.309420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c5da0-a2ee-4dff-9715-1b222ca006bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:03.310953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c5da0-a2ee-4dff-9715-1b222ca006bd service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:03.320663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 723/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:05.650801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65cf9c-d78c-4fc5-8802-c74a8ab0fc56 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:05.651400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65cf9c-d78c-4fc5-8802-c74a8ab0fc56 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:05.651400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65cf9c-d78c-4fc5-8802-c74a8ab0fc56 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:05.652734 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:05.658242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a33090-7936-438e-98d4-c1bd0c78d1f4 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:05.659248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a33090-7936-438e-98d4-c1bd0c78d1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:05.659412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a33090-7936-438e-98d4-c1bd0c78d1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:05.676764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-68a33090-7936-438e-98d4-c1bd0c78d1f4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:05.677940 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-68a33090-7936-438e-98d4-c1bd0c78d1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:05.685571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 12:53:05.724046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65cf9c-d78c-4fc5-8802-c74a8ab0fc56 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:05.734644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65cf9c-d78c-4fc5-8802-c74a8ab0fc56 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:05.738155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b29780-e8bc-4efd-bd3c-50a699d3d7b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:05.738847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b29780-e8bc-4efd-bd3c-50a699d3d7b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:05.739117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b29780-e8bc-4efd-bd3c-50a699d3d7b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:05.739396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b29780-e8bc-4efd-bd3c-50a699d3d7b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:05.748159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b29780-e8bc-4efd-bd3c-50a699d3d7b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:05.749590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b29780-e8bc-4efd-bd3c-50a699d3d7b0 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:05.759155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 728/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:53:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:08.000511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333cb63-5f7b-4c53-a469-56adfcdf935f None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:08.001086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333cb63-5f7b-4c53-a469-56adfcdf935f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:08.001086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333cb63-5f7b-4c53-a469-56adfcdf935f None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:08.002502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:08.007944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553528b-0d93-4459-89bb-e2d32f496ef8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:08.008382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553528b-0d93-4459-89bb-e2d32f496ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:08.008480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553528b-0d93-4459-89bb-e2d32f496ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:08.027055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c553528b-0d93-4459-89bb-e2d32f496ef8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:08.028233 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c553528b-0d93-4459-89bb-e2d32f496ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:08.036496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:53:08.073514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333cb63-5f7b-4c53-a469-56adfcdf935f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:08.084681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333cb63-5f7b-4c53-a469-56adfcdf935f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:08.088190 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1150a-dc89-4457-8de7-534d6aa4ad81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:08.088825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1150a-dc89-4457-8de7-534d6aa4ad81 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:08.089110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1150a-dc89-4457-8de7-534d6aa4ad81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:08.089389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1150a-dc89-4457-8de7-534d6aa4ad81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:08.098120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1150a-dc89-4457-8de7-534d6aa4ad81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:08.099569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1150a-dc89-4457-8de7-534d6aa4ad81 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:08.108688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 726/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:10.342478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d9533-e3e8-4975-878e-e57ec7370991 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:10.343067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d9533-e3e8-4975-878e-e57ec7370991 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:10.343067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d9533-e3e8-4975-878e-e57ec7370991 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:10.344450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:10.349767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27009e-a043-4b6c-a6b4-a2cf04b0760b None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:10.350184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27009e-a043-4b6c-a6b4-a2cf04b0760b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:10.350362 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27009e-a043-4b6c-a6b4-a2cf04b0760b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:10.368436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-0a27009e-a043-4b6c-a6b4-a2cf04b0760b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:10.369622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-0a27009e-a043-4b6c-a6b4-a2cf04b0760b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:10.377811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 12:53:10.415258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d9533-e3e8-4975-878e-e57ec7370991 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:10.425767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d9533-e3e8-4975-878e-e57ec7370991 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:10.429210 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6b71a-d4eb-4d24-ac42-4d3f16580d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:10.429840 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6b71a-d4eb-4d24-ac42-4d3f16580d9e service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:10.430121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6b71a-d4eb-4d24-ac42-4d3f16580d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:10.430424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6b71a-d4eb-4d24-ac42-4d3f16580d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:10.438992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6b71a-d4eb-4d24-ac42-4d3f16580d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:10.440907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6b71a-d4eb-4d24-ac42-4d3f16580d9e service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:10.449982 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 731/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:53:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:12.677615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd087f9-e999-4d60-9e0d-59194ccda8de None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:12.678124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd087f9-e999-4d60-9e0d-59194ccda8de None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:12.678124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd087f9-e999-4d60-9e0d-59194ccda8de None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:12.679483 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:12.684757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a4ddb-5aa4-4468-9610-7138bfc70b74 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:12.685186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a4ddb-5aa4-4468-9610-7138bfc70b74 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:12.685311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a4ddb-5aa4-4468-9610-7138bfc70b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:12.691397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=70 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:53:12.705350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6e5a4ddb-5aa4-4468-9610-7138bfc70b74 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:12.706598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6e5a4ddb-5aa4-4468-9610-7138bfc70b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:12.714654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:53:12.751242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd087f9-e999-4d60-9e0d-59194ccda8de None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:12.756953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=73 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:53:12.764049 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd087f9-e999-4d60-9e0d-59194ccda8de None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:12.767619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6cac2-cccc-48d8-83b6-94a9995aadb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:12.768260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6cac2-cccc-48d8-83b6-94a9995aadb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:12.768547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6cac2-cccc-48d8-83b6-94a9995aadb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:12.768827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6cac2-cccc-48d8-83b6-94a9995aadb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:12.777802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6cac2-cccc-48d8-83b6-94a9995aadb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:12.779254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6cac2-cccc-48d8-83b6-94a9995aadb9 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:12.789105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 729/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:53:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:15.037046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebc4ce-e863-4a12-91c5-d871ee53e418 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:15.037573 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebc4ce-e863-4a12-91c5-d871ee53e418 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:15.037573 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebc4ce-e863-4a12-91c5-d871ee53e418 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:15.038902 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:15.044309 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d23f-995f-4173-a96a-0b674f7a151d None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:15.044725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d23f-995f-4173-a96a-0b674f7a151d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:15.044870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d23f-995f-4173-a96a-0b674f7a151d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:15.063099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-f955d23f-995f-4173-a96a-0b674f7a151d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:15.064298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f955d23f-995f-4173-a96a-0b674f7a151d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:15.077341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:53:15.115421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebc4ce-e863-4a12-91c5-d871ee53e418 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:15.126150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebc4ce-e863-4a12-91c5-d871ee53e418 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:15.129912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b34564-a57d-4ccd-880d-af204ab8f538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:15.130578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b34564-a57d-4ccd-880d-af204ab8f538 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:15.130866 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b34564-a57d-4ccd-880d-af204ab8f538 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:15.131141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b34564-a57d-4ccd-880d-af204ab8f538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:15.139938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b34564-a57d-4ccd-880d-af204ab8f538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:15.141383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b34564-a57d-4ccd-880d-af204ab8f538 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:15.151273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 734/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:17.340174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba46d6-ce0e-400b-a006-1fa9947acb02 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:17.340739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba46d6-ce0e-400b-a006-1fa9947acb02 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:17.340739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba46d6-ce0e-400b-a006-1fa9947acb02 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:17.342093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:17.347802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08584aaa-bee2-4672-a3a0-d22a47703e9d None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:17.348238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08584aaa-bee2-4672-a3a0-d22a47703e9d None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:17.348333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08584aaa-bee2-4672-a3a0-d22a47703e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:17.366068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-08584aaa-bee2-4672-a3a0-d22a47703e9d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:17.367283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-08584aaa-bee2-4672-a3a0-d22a47703e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:17.374913 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:53:17.411883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba46d6-ce0e-400b-a006-1fa9947acb02 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:17.422450 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba46d6-ce0e-400b-a006-1fa9947acb02 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:17.426033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021d50a-982a-44cd-8e5e-c054322af8bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:17.426738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021d50a-982a-44cd-8e5e-c054322af8bf service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:17.427018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021d50a-982a-44cd-8e5e-c054322af8bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:17.427290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021d50a-982a-44cd-8e5e-c054322af8bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:17.436176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021d50a-982a-44cd-8e5e-c054322af8bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:17.437586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021d50a-982a-44cd-8e5e-c054322af8bf service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:17.446660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 732/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:19.637992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280e4e-37bd-4b5c-98f8-70eb89fb0afa None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:19.638582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280e4e-37bd-4b5c-98f8-70eb89fb0afa None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:19.638582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280e4e-37bd-4b5c-98f8-70eb89fb0afa None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:19.639957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:19.645919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c0b82-eb60-4655-ae72-935e22545c61 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:19.646362 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c0b82-eb60-4655-ae72-935e22545c61 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:19.646503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c0b82-eb60-4655-ae72-935e22545c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:19.663690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-f91c0b82-eb60-4655-ae72-935e22545c61 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:19.664802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f91c0b82-eb60-4655-ae72-935e22545c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:19.672310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:53:19.709261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f280e4e-37bd-4b5c-98f8-70eb89fb0afa None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:19.719791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f280e4e-37bd-4b5c-98f8-70eb89fb0afa None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:19.723333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3612dd-cfe2-4382-8b6f-a6205742cfa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:19.723972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3612dd-cfe2-4382-8b6f-a6205742cfa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:19.724259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3612dd-cfe2-4382-8b6f-a6205742cfa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:19.724544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3612dd-cfe2-4382-8b6f-a6205742cfa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:19.733262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3612dd-cfe2-4382-8b6f-a6205742cfa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:19.734749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3612dd-cfe2-4382-8b6f-a6205742cfa5 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:19.744498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 737/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:53:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:29.658645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=17 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:53:29.740417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=17 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:53:52.618504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae1ff9-4d95-43b8-9bf6-6f3ea91a7864 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:52.619097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae1ff9-4d95-43b8-9bf6-6f3ea91a7864 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:52.619097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae1ff9-4d95-43b8-9bf6-6f3ea91a7864 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:52.620432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 734/1471] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:52.625651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f34da0-8fa5-42fe-ab1d-4171cdad46de None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:52.626099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f34da0-8fa5-42fe-ab1d-4171cdad46de None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:52.626208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f34da0-8fa5-42fe-ab1d-4171cdad46de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:52.644945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-36f34da0-8fa5-42fe-ab1d-4171cdad46de None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:52.646133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-36f34da0-8fa5-42fe-ab1d-4171cdad46de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:52.653827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 738/1472] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:53:52.699998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b150af3-9b5e-4b5f-a3c7-0f6118f332c9 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:52.700422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b150af3-9b5e-4b5f-a3c7-0f6118f332c9 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:52.700733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b150af3-9b5e-4b5f-a3c7-0f6118f332c9 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:52.702266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 735/1473] 10.222.0.45 () {54 vars in 962 bytes} [Tue May 24 12:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:52.707216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4f218-e08f-4c37-a973-4d5d5ff91194 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:52.707536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4f218-e08f-4c37-a973-4d5d5ff91194 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:52.707818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4f218-e08f-4c37-a973-4d5d5ff91194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:52.725387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-32f4f218-e08f-4c37-a973-4d5d5ff91194 None None] MFA Rules not processed for user `e47f1d65687e4cfcbce764076141702c`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:52.726587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-32f4f218-e08f-4c37-a973-4d5d5ff91194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:52.732798 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 739/1474] 10.222.0.45 () {58 vars in 1059 bytes} [Tue May 24 12:53:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:53:52.740482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b150af3-9b5e-4b5f-a3c7-0f6118f332c9 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:52.750747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b150af3-9b5e-4b5f-a3c7-0f6118f332c9 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:52.755072 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c035c6b-be28-4d15-9e32-f3f3309d96db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e47f1d65687e4cfcbce764076141702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:52.755696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c035c6b-be28-4d15-9e32-f3f3309d96db service glance] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:52.755978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c035c6b-be28-4d15-9e32-f3f3309d96db service glance] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:52.756250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c035c6b-be28-4d15-9e32-f3f3309d96db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:52.764848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c035c6b-be28-4d15-9e32-f3f3309d96db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:52.766251 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c035c6b-be28-4d15-9e32-f3f3309d96db service glance] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:52.775443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 736/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:53:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:55.041013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa43450-fcc6-42aa-b95e-52634f80a162 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:55.041584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa43450-fcc6-42aa-b95e-52634f80a162 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:55.041584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa43450-fcc6-42aa-b95e-52634f80a162 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:55.042946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 740/1476] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:55.048169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377b225-18e6-4916-a5cd-8b3c26e3c305 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:55.049055 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377b225-18e6-4916-a5cd-8b3c26e3c305 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:55.049265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377b225-18e6-4916-a5cd-8b3c26e3c305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:55.067116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-b377b225-18e6-4916-a5cd-8b3c26e3c305 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:55.068289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b377b225-18e6-4916-a5cd-8b3c26e3c305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:55.075842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 737/1477] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:53:55.099624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa43450-fcc6-42aa-b95e-52634f80a162 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:55.111807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa43450-fcc6-42aa-b95e-52634f80a162 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:55.116217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82004e05-6f03-4522-bd4e-db21397be6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:55.116862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004e05-6f03-4522-bd4e-db21397be6ce admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:55.117148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004e05-6f03-4522-bd4e-db21397be6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:55.117427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004e05-6f03-4522-bd4e-db21397be6ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:55.118159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82004e05-6f03-4522-bd4e-db21397be6ce admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:55.120087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: WARNING py.warnings [None req-82004e05-6f03-4522-bd4e-db21397be6ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 12:53:55.120087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: warnings.warn(msg) May 24 12:53:55.120087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: May 24 12:53:55.121220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82004e05-6f03-4522-bd4e-db21397be6ce admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:55.140009 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 741/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 12:53:55 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:53:57.373051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47ec2f-deb1-4178-a8e1-54315bb457d7 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:57.373620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47ec2f-deb1-4178-a8e1-54315bb457d7 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:57.373620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47ec2f-deb1-4178-a8e1-54315bb457d7 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:57.375019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 738/1479] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:57.380702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3054-e964-461f-ac96-8a0ca6965da8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:57.381147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3054-e964-461f-ac96-8a0ca6965da8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:57.381250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3054-e964-461f-ac96-8a0ca6965da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:57.399584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-5c7e3054-e964-461f-ac96-8a0ca6965da8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:57.400795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5c7e3054-e964-461f-ac96-8a0ca6965da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:57.409297 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 742/1480] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 12:53:57.451109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47ec2f-deb1-4178-a8e1-54315bb457d7 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:53:57.465893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47ec2f-deb1-4178-a8e1-54315bb457d7 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:53:57.471770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa16adb-e80d-4d23-b58a-15e416009497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:53:57.472407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa16adb-e80d-4d23-b58a-15e416009497 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:57.472690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa16adb-e80d-4d23-b58a-15e416009497 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:57.472967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa16adb-e80d-4d23-b58a-15e416009497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:57.480880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa16adb-e80d-4d23-b58a-15e416009497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:53:57.482455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa16adb-e80d-4d23-b58a-15e416009497 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:53:57.491610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 739/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:53:59.937057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb9450-2b68-46d4-93e3-237e7cef346f None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:59.937597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb9450-2b68-46d4-93e3-237e7cef346f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:59.937597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb9450-2b68-46d4-93e3-237e7cef346f None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:59.939018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 743/1482] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:53:59.944268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74338371-06ec-4337-968f-0c824ab9fd31 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:53:59.944692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74338371-06ec-4337-968f-0c824ab9fd31 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:53:59.944811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74338371-06ec-4337-968f-0c824ab9fd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:53:59.962206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-74338371-06ec-4337-968f-0c824ab9fd31 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:53:59.963426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-74338371-06ec-4337-968f-0c824ab9fd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:53:59.971632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 740/1483] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:54:00.011721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb9450-2b68-46d4-93e3-237e7cef346f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:54:00.022493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb9450-2b68-46d4-93e3-237e7cef346f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:54:00.025976 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae887043-04a5-4047-8871-e2ce210dacd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:54:00.026674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887043-04a5-4047-8871-e2ce210dacd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:54:00.026962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887043-04a5-4047-8871-e2ce210dacd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:54:00.027238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887043-04a5-4047-8871-e2ce210dacd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:54:00.036640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae887043-04a5-4047-8871-e2ce210dacd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:54:00.038084 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae887043-04a5-4047-8871-e2ce210dacd0 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:54:00.047324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 744/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:54:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:54:02.277693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3b92d-f125-4fb0-b77d-b062770a13cf None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:54:02.278275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3b92d-f125-4fb0-b77d-b062770a13cf None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:54:02.278275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3b92d-f125-4fb0-b77d-b062770a13cf None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:54:02.279704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 741/1485] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:54:02.285022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e2c43-592e-49c6-aff7-694a87691d02 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:54:02.285461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e2c43-592e-49c6-aff7-694a87691d02 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:54:02.285613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e2c43-592e-49c6-aff7-694a87691d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:54:02.303097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-bc6e2c43-592e-49c6-aff7-694a87691d02 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:54:02.304290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bc6e2c43-592e-49c6-aff7-694a87691d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:54:02.312022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 745/1486] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:54:02.349589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3b92d-f125-4fb0-b77d-b062770a13cf None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:54:02.360220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3b92d-f125-4fb0-b77d-b062770a13cf None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:54:02.363697 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341c5cf-2747-4a30-8d03-f0fb9f1b7dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:54:02.364340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341c5cf-2747-4a30-8d03-f0fb9f1b7dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:54:02.364624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341c5cf-2747-4a30-8d03-f0fb9f1b7dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:54:02.364900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341c5cf-2747-4a30-8d03-f0fb9f1b7dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:54:02.373671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341c5cf-2747-4a30-8d03-f0fb9f1b7dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:54:02.375174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341c5cf-2747-4a30-8d03-f0fb9f1b7dc8 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:54:02.386097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 742/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:54:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:54:04.700868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717321d-52db-4373-9c5e-5706599307ed None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:54:04.701379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717321d-52db-4373-9c5e-5706599307ed None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:54:04.701379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717321d-52db-4373-9c5e-5706599307ed None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:54:04.702742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 746/1488] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:54:04.708549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b28c2-1094-42c3-b165-49ab0e703885 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:54:04.708947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b28c2-1094-42c3-b165-49ab0e703885 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:54:04.709134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b28c2-1094-42c3-b165-49ab0e703885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:54:04.727222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e66b28c2-1094-42c3-b165-49ab0e703885 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:54:04.728403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e66b28c2-1094-42c3-b165-49ab0e703885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:54:04.736027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 743/1489] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:54:04.775364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717321d-52db-4373-9c5e-5706599307ed None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:54:04.786657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717321d-52db-4373-9c5e-5706599307ed None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:54:04.790404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8b086-f19c-4ca3-b0a6-7a58bc0d523f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:54:04.791033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8b086-f19c-4ca3-b0a6-7a58bc0d523f service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:54:04.791319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8b086-f19c-4ca3-b0a6-7a58bc0d523f service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:54:04.791591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8b086-f19c-4ca3-b0a6-7a58bc0d523f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:54:04.800263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8b086-f19c-4ca3-b0a6-7a58bc0d523f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:54:04.801688 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8b086-f19c-4ca3-b0a6-7a58bc0d523f service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:54:04.810799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 747/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 12:54:07.180630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30a7dc-4c59-4876-bc8a-76fc0b5a3bd4 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:54:07.181235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30a7dc-4c59-4876-bc8a-76fc0b5a3bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:54:07.181235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30a7dc-4c59-4876-bc8a-76fc0b5a3bd4 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:54:07.182585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 744/1491] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 12:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:54:07.187796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebe398-b7e0-409e-aced-83200194cfbd None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:54:07.188254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebe398-b7e0-409e-aced-83200194cfbd None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:54:07.188373 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebe398-b7e0-409e-aced-83200194cfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:54:07.206638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f9ebe398-b7e0-409e-aced-83200194cfbd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:54:07.207876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f9ebe398-b7e0-409e-aced-83200194cfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:54:07.216235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 748/1492] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 12:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:54:07.261996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30a7dc-4c59-4876-bc8a-76fc0b5a3bd4 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:54:07.272521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30a7dc-4c59-4876-bc8a-76fc0b5a3bd4 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:54:07.275980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d2a8e-eb4e-42a7-8663-a36fc92577f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:54:07.276611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d2a8e-eb4e-42a7-8663-a36fc92577f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:54:07.276893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d2a8e-eb4e-42a7-8663-a36fc92577f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:54:07.277167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d2a8e-eb4e-42a7-8663-a36fc92577f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:54:07.286297 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d2a8e-eb4e-42a7-8663-a36fc92577f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:54:07.287715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d2a8e-eb4e-42a7-8663-a36fc92577f4 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:54:07.296950 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 745/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:54:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:54:17.201547 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=33 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:54:17.292794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=33 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:56:15.918831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03155ca-9c37-4721-9606-65027e4df492 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:15.919432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03155ca-9c37-4721-9606-65027e4df492 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:15.919432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03155ca-9c37-4721-9606-65027e4df492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:15.937883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b03155ca-9c37-4721-9606-65027e4df492 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:56:15.939078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b03155ca-9c37-4721-9606-65027e4df492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:56:15.946647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 749/1494] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:56:15.955667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def283d-6f35-4e5a-ba55-1854bfd72cc6 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:15.956157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def283d-6f35-4e5a-ba55-1854bfd72cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:15.956253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def283d-6f35-4e5a-ba55-1854bfd72cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:15.973519 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-2def283d-6f35-4e5a-ba55-1854bfd72cc6 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:56:15.974724 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2def283d-6f35-4e5a-ba55-1854bfd72cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:56:15.983079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 746/1495] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:56:15.987323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03155ca-9c37-4721-9606-65027e4df492 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:56:15.996577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03155ca-9c37-4721-9606-65027e4df492 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:56:16.001561 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf019dd-f087-4529-83b8-bad3d265f196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:56:16.002490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf019dd-f087-4529-83b8-bad3d265f196 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.002777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf019dd-f087-4529-83b8-bad3d265f196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.003051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf019dd-f087-4529-83b8-bad3d265f196 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.003764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf019dd-f087-4529-83b8-bad3d265f196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:56:16.004066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf019dd-f087-4529-83b8-bad3d265f196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:56:16.005079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: WARNING py.warnings [None req-1cf019dd-f087-4529-83b8-bad3d265f196 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 12:56:16.005079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: warnings.warn(msg) May 24 12:56:16.005079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: May 24 12:56:16.005951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf019dd-f087-4529-83b8-bad3d265f196 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:56:16.017017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 750/1496] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 12:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:56:16.024848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def283d-6f35-4e5a-ba55-1854bfd72cc6 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:56:16.033678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def283d-6f35-4e5a-ba55-1854bfd72cc6 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:56:16.038086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9d5b1-bb74-4c6f-a2f5-31db440fc168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:56:16.038762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9d5b1-bb74-4c6f-a2f5-31db440fc168 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.039053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9d5b1-bb74-4c6f-a2f5-31db440fc168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.039327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9d5b1-bb74-4c6f-a2f5-31db440fc168 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.040016 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9d5b1-bb74-4c6f-a2f5-31db440fc168 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:56:16.041757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: WARNING py.warnings [None req-dee9d5b1-bb74-4c6f-a2f5-31db440fc168 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 12:56:16.041757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: warnings.warn(msg) May 24 12:56:16.041757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: May 24 12:56:16.042972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9d5b1-bb74-4c6f-a2f5-31db440fc168 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:56:16.079906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 747/1497] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 12:56:16 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:56:16.084317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf019dd-f087-4529-83b8-bad3d265f196 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:56:16.094000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf019dd-f087-4529-83b8-bad3d265f196 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:56:16.147833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6639bb4-3a51-4fd0-b645-8fb6ef0ca797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:56:16.148520 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6639bb4-3a51-4fd0-b645-8fb6ef0ca797 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.148801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6639bb4-3a51-4fd0-b645-8fb6ef0ca797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.149076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6639bb4-3a51-4fd0-b645-8fb6ef0ca797 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.149770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6639bb4-3a51-4fd0-b645-8fb6ef0ca797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:56:16.150814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: WARNING py.warnings [None req-c6639bb4-3a51-4fd0-b645-8fb6ef0ca797 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 12:56:16.150814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: warnings.warn(msg) May 24 12:56:16.150814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: May 24 12:56:16.152027 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6639bb4-3a51-4fd0-b645-8fb6ef0ca797 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:56:16.170652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 751/1498] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 12:56:16 2022] POST /identity/v3/users => generated 289 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:56:16.174618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9d5b1-bb74-4c6f-a2f5-31db440fc168 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:56:16.184631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9d5b1-bb74-4c6f-a2f5-31db440fc168 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:56:16.189635 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45641c1e-3255-4c18-8c1a-7204b7ac4c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:56:16.190306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45641c1e-3255-4c18-8c1a-7204b7ac4c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.190621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45641c1e-3255-4c18-8c1a-7204b7ac4c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.190884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45641c1e-3255-4c18-8c1a-7204b7ac4c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.191606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45641c1e-3255-4c18-8c1a-7204b7ac4c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:56:16.192609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: WARNING py.warnings [None req-45641c1e-3255-4c18-8c1a-7204b7ac4c2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 12:56:16.192609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: warnings.warn(msg) May 24 12:56:16.192609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: May 24 12:56:16.193484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45641c1e-3255-4c18-8c1a-7204b7ac4c2b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:56:16.205981 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 748/1499] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:56:16.210183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6639bb4-3a51-4fd0-b645-8fb6ef0ca797 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:56:16.221012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6639bb4-3a51-4fd0-b645-8fb6ef0ca797 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:56:16.226782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10881551-124a-4361-b6ce-e84286de0abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:56:16.227543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10881551-124a-4361-b6ce-e84286de0abf admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.227903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10881551-124a-4361-b6ce-e84286de0abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.228279 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10881551-124a-4361-b6ce-e84286de0abf admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.229150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10881551-124a-4361-b6ce-e84286de0abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:56:16.230489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: WARNING py.warnings [None req-10881551-124a-4361-b6ce-e84286de0abf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 12:56:16.230489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: warnings.warn(msg) May 24 12:56:16.230489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: May 24 12:56:16.231599 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10881551-124a-4361-b6ce-e84286de0abf admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:56:16.246244 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 752/1500] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 12:56:16.251172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45641c1e-3255-4c18-8c1a-7204b7ac4c2b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:56:16.261186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45641c1e-3255-4c18-8c1a-7204b7ac4c2b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:56:16.265655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a6fb3-cb02-46cf-b7c2-169a7407057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:56:16.266570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a6fb3-cb02-46cf-b7c2-169a7407057d admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.266853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a6fb3-cb02-46cf-b7c2-169a7407057d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.267127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a6fb3-cb02-46cf-b7c2-169a7407057d admin admin] PATH_INFO: `/v3/projects/dd806d72618b4e048986f040799c9995/users/205427f2449445d2b7f676fb06a879b6/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.282014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a6fb3-cb02-46cf-b7c2-169a7407057d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd806d72618b4e048986f040799c9995, user_id=205427f2449445d2b7f676fb06a879b6, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:56:16.283103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: WARNING py.warnings [None req-c59a6fb3-cb02-46cf-b7c2-169a7407057d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 12:56:16.283103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: warnings.warn(msg) May 24 12:56:16.283103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: May 24 12:56:16.285042 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a6fb3-cb02-46cf-b7c2-169a7407057d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:56:16.297950 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 749/1501] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:56:16 2022] PUT /identity/v3/projects/dd806d72618b4e048986f040799c9995/users/205427f2449445d2b7f676fb06a879b6/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:56:16.308951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918c2db-d9f1-403b-94f5-ec3de3ad6ea4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.309327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918c2db-d9f1-403b-94f5-ec3de3ad6ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.309609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918c2db-d9f1-403b-94f5-ec3de3ad6ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.335530 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-1918c2db-d9f1-403b-94f5-ec3de3ad6ea4 None None] MFA Rules not processed for user `205427f2449445d2b7f676fb06a879b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:56:16.336732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1918c2db-d9f1-403b-94f5-ec3de3ad6ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:56:16.400078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 753/1502] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:56:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:56:16.414699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e731c9-b2d8-43ee-a5cb-f3b0b2b87fbb None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.415019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e731c9-b2d8-43ee-a5cb-f3b0b2b87fbb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.415304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e731c9-b2d8-43ee-a5cb-f3b0b2b87fbb None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.416679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 750/1503] 10.222.0.45 () {54 vars in 963 bytes} [Tue May 24 12:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:56:16.422306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a1fdd-a1b4-4d3e-b4fc-654a26e2e5a4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.422555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a1fdd-a1b4-4d3e-b4fc-654a26e2e5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.422836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a1fdd-a1b4-4d3e-b4fc-654a26e2e5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.440508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b92a1fdd-a1b4-4d3e-b4fc-654a26e2e5a4 None None] MFA Rules not processed for user `8c9f08515d514be8af1176717ec9e6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:56:16.441702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b92a1fdd-a1b4-4d3e-b4fc-654a26e2e5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:56:16.484550 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:56:16.492681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 754/1504] 10.222.0.45 () {58 vars in 1060 bytes} [Tue May 24 12:56:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:56:16.500377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e731c9-b2d8-43ee-a5cb-f3b0b2b87fbb None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:56:16.510858 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e731c9-b2d8-43ee-a5cb-f3b0b2b87fbb None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:56:16.514566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69134c-b116-4e20-8829-0811e3705f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:56:16.515203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69134c-b116-4e20-8829-0811e3705f74 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.515486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69134c-b116-4e20-8829-0811e3705f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.515762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69134c-b116-4e20-8829-0811e3705f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.523996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69134c-b116-4e20-8829-0811e3705f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:56:16.525412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69134c-b116-4e20-8829-0811e3705f74 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:56:16.534450 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 751/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:56:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:56:16.575325 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba59349-6906-44fe-a3d1-7093aee5951b None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.575613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba59349-6906-44fe-a3d1-7093aee5951b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.575921 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba59349-6906-44fe-a3d1-7093aee5951b None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.577194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 755/1506] 10.222.0.45 () {52 vars in 829 bytes} [Tue May 24 12:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:56:16.580714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69134c-b116-4e20-8829-0811e3705f74 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:56:16.641557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69134c-b116-4e20-8829-0811e3705f74 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:56:16.646047 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:56:16.646937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.647221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.647497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] PATH_INFO: `/v3/users/205427f2449445d2b7f676fb06a879b6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.648670 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=205427f2449445d2b7f676fb06a879b6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:56:16.649678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: WARNING py.warnings [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 12:56:16.649678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: warnings.warn(msg) May 24 12:56:16.649678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: May 24 12:56:16.650907 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:56:16.712811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '205427f2449445d2b7f676fb06a879b6'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 12:56:16.731188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '205427f2449445d2b7f676fb06a879b6'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 12:56:16.740482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '205427f2449445d2b7f676fb06a879b6'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 12:56:16.741096 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '205427f2449445d2b7f676fb06a879b6'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 12:56:16.749629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:56:16.755184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '205427f2449445d2b7f676fb06a879b6'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 12:56:16.762104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 752/1507] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 12:56:16 2022] DELETE /identity/v3/users/205427f2449445d2b7f676fb06a879b6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:56:16.767038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef893e2-9504-473c-b4e9-6cc157826c1a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.767334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef893e2-9504-473c-b4e9-6cc157826c1a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.767612 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef893e2-9504-473c-b4e9-6cc157826c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.785569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-cef893e2-9504-473c-b4e9-6cc157826c1a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:56:16.786799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-cef893e2-9504-473c-b4e9-6cc157826c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:56:16.843504 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 756/1508] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:56:16.853548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:56:16.854933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:56:16.911163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd983be-1bfd-4141-9657-6f57e7f57bf1 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:56:16.914791 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efde0d-2885-44d7-b81a-8cf9e8153695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:56:16.915438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efde0d-2885-44d7-b81a-8cf9e8153695 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:16.915726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efde0d-2885-44d7-b81a-8cf9e8153695 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:16.916008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efde0d-2885-44d7-b81a-8cf9e8153695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:16.924822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efde0d-2885-44d7-b81a-8cf9e8153695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:56:16.926259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efde0d-2885-44d7-b81a-8cf9e8153695 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:56:16.935541 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 753/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:56:17.391145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef893e2-9504-473c-b4e9-6cc157826c1a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:56:17.392677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-cef893e2-9504-473c-b4e9-6cc157826c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:56:17.410163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef893e2-9504-473c-b4e9-6cc157826c1a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:56:17.414881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:56:17.415752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:56:17.416030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:56:17.416304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] PATH_INFO: `/v3/projects/dd806d72618b4e048986f040799c9995` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:56:17.417496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd806d72618b4e048986f040799c9995)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:56:17.418580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: WARNING py.warnings [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 12:56:17.418580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: warnings.warn(msg) May 24 12:56:17.418580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: May 24 12:56:17.420439 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:56:17.481293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd806d72618b4e048986f040799c9995', 'request_id': 'req-11492b72-40fd-48dd-ab44-70e3ad23ffd5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 12:56:17.491563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd806d72618b4e048986f040799c9995', 'request_id': 'req-11492b72-40fd-48dd-ab44-70e3ad23ffd5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 12:56:17.498569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd806d72618b4e048986f040799c9995', 'request_id': 'req-11492b72-40fd-48dd-ab44-70e3ad23ffd5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 12:56:17.516696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] The token cache is being invalidate because project dd806d72618b4e048986f040799c9995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 12:56:17.516983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 12:56:17.517716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-11492b72-40fd-48dd-ab44-70e3ad23ffd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 12:56:17.519290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 757/1510] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 12:56:17 2022] DELETE /identity/v3/projects/dd806d72618b4e048986f040799c9995 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 12:56:26.932939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=29 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:56:27.513673 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:57:57.166225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bc458-d442-44a9-a9ae-30e73486cdcb None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:57:57.166817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bc458-d442-44a9-a9ae-30e73486cdcb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:57:57.166817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bc458-d442-44a9-a9ae-30e73486cdcb None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:57:57.168146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:57:57.173049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd63c-5b1f-488e-aacd-1050ecbdbb4c None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:57:57.173493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd63c-5b1f-488e-aacd-1050ecbdbb4c None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:57:57.173615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd63c-5b1f-488e-aacd-1050ecbdbb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:57:57.191677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e39fd63c-5b1f-488e-aacd-1050ecbdbb4c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:57:57.192891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e39fd63c-5b1f-488e-aacd-1050ecbdbb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:57:57.251031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:57:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:57:57.416626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bc458-d442-44a9-a9ae-30e73486cdcb None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:57:57.420186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c90bc458-d442-44a9-a9ae-30e73486cdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:57:57.494812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bc458-d442-44a9-a9ae-30e73486cdcb None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:57:57.501099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6eef91-464a-4e7e-aacc-e0bdb35ef462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:57:57.501748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6eef91-464a-4e7e-aacc-e0bdb35ef462 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:57:57.502034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6eef91-464a-4e7e-aacc-e0bdb35ef462 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:57:57.502340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6eef91-464a-4e7e-aacc-e0bdb35ef462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:57:57.510321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6eef91-464a-4e7e-aacc-e0bdb35ef462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:57:57.512437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6eef91-464a-4e7e-aacc-e0bdb35ef462 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:57:57.521773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 755/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:57:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:57:58.404610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed84616-a0f2-4dc9-986b-0686a3842dd8 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:57:58.405186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed84616-a0f2-4dc9-986b-0686a3842dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:57:58.405186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed84616-a0f2-4dc9-986b-0686a3842dd8 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:57:58.406592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:57:58.411911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d78d3-db58-4159-87ba-d119b51693a5 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:57:58.412346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d78d3-db58-4159-87ba-d119b51693a5 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:57:58.412477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d78d3-db58-4159-87ba-d119b51693a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:57:58.432110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-926d78d3-db58-4159-87ba-d119b51693a5 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:57:58.433850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-926d78d3-db58-4159-87ba-d119b51693a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:57:58.441835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:57:58.594892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed84616-a0f2-4dc9-986b-0686a3842dd8 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:57:58.608898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed84616-a0f2-4dc9-986b-0686a3842dd8 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:57:58.614321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca573e-0804-40f0-9ba8-253692d6a6c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:57:58.614957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca573e-0804-40f0-9ba8-253692d6a6c3 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:57:58.615243 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca573e-0804-40f0-9ba8-253692d6a6c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:57:58.615523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca573e-0804-40f0-9ba8-253692d6a6c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:57:58.624711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca573e-0804-40f0-9ba8-253692d6a6c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:57:58.626663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca573e-0804-40f0-9ba8-253692d6a6c3 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:57:58.636781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 760/1516] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:57:59.038111 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94346797-f45c-452f-9dcb-d2ff61dd8c77 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:57:59.038350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94346797-f45c-452f-9dcb-d2ff61dd8c77 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:57:59.038630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94346797-f45c-452f-9dcb-d2ff61dd8c77 None None] PATH_INFO: `/v3` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:57:59.039935 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 757/1517] 10.222.0.13 () {54 vars in 925 bytes} [Tue May 24 12:57:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:57:59.046171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298f30e-0730-4ef4-affc-ef2484821fdb None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:57:59.046492 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298f30e-0730-4ef4-affc-ef2484821fdb None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:57:59.046776 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298f30e-0730-4ef4-affc-ef2484821fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:57:59.064467 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-2298f30e-0730-4ef4-affc-ef2484821fdb None None] MFA Rules not processed for user `630f0527bd844de9be072c459dab96c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:57:59.065668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2298f30e-0730-4ef4-affc-ef2484821fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:57:59.125950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 761/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Tue May 24 12:57:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:58:03.007317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad1ef3-c5d4-4f16-bc36-ef16b2a08a2f None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:03.007873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad1ef3-c5d4-4f16-bc36-ef16b2a08a2f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:03.007873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad1ef3-c5d4-4f16-bc36-ef16b2a08a2f None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:03.009244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 758/1519] 10.222.0.45 () {54 vars in 960 bytes} [Tue May 24 12:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:58:03.014928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e8302-53d0-43fd-9171-e00b37f94a0f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:03.015338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e8302-53d0-43fd-9171-e00b37f94a0f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:03.015482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e8302-53d0-43fd-9171-e00b37f94a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:03.035178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-136e8302-53d0-43fd-9171-e00b37f94a0f None None] MFA Rules not processed for user `630f0527bd844de9be072c459dab96c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:58:03.036306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-136e8302-53d0-43fd-9171-e00b37f94a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:58:03.044790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 762/1520] 10.222.0.45 () {58 vars in 1057 bytes} [Tue May 24 12:58:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:58:03.052350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad1ef3-c5d4-4f16-bc36-ef16b2a08a2f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:58:03.065969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad1ef3-c5d4-4f16-bc36-ef16b2a08a2f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:58:03.070835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ca36a-f600-4c88-85e7-11f150ebf1b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:58:03.071470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ca36a-f600-4c88-85e7-11f150ebf1b0 service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:03.071755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ca36a-f600-4c88-85e7-11f150ebf1b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:03.072031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ca36a-f600-4c88-85e7-11f150ebf1b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:03.080668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ca36a-f600-4c88-85e7-11f150ebf1b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:58:03.082237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ca36a-f600-4c88-85e7-11f150ebf1b0 service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:58:03.091759 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 759/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:58:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:58:09.906093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d007a3-38ca-452e-ba3c-c171e799cbf9 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:09.906700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d007a3-38ca-452e-ba3c-c171e799cbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:09.906700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d007a3-38ca-452e-ba3c-c171e799cbf9 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:09.908041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:58:09.912715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4731d23-4bec-46d5-9531-59ef19b8ba85 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:09.913162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4731d23-4bec-46d5-9531-59ef19b8ba85 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:09.913268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4731d23-4bec-46d5-9531-59ef19b8ba85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:09.930681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e4731d23-4bec-46d5-9531-59ef19b8ba85 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:58:09.931852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e4731d23-4bec-46d5-9531-59ef19b8ba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:58:09.939500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:58:10.089221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d007a3-38ca-452e-ba3c-c171e799cbf9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:58:10.104092 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d007a3-38ca-452e-ba3c-c171e799cbf9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:58:10.109624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeeb2aa-cd9d-4d1d-ad92-d9da72fb039e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:58:10.110263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeeb2aa-cd9d-4d1d-ad92-d9da72fb039e service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:10.110579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeeb2aa-cd9d-4d1d-ad92-d9da72fb039e service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:10.110858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeeb2aa-cd9d-4d1d-ad92-d9da72fb039e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:10.118879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeeb2aa-cd9d-4d1d-ad92-d9da72fb039e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:58:10.120492 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeeb2aa-cd9d-4d1d-ad92-d9da72fb039e service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:58:10.130086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 764/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:58:10.991915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb85259-3729-4b8e-8f0b-6fed26169317 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:10.992468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb85259-3729-4b8e-8f0b-6fed26169317 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:10.992468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb85259-3729-4b8e-8f0b-6fed26169317 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:10.993861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 12:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:58:10.998771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecce5b2-a617-454f-8247-8c912529669e None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:10.999164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecce5b2-a617-454f-8247-8c912529669e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:10.999332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecce5b2-a617-454f-8247-8c912529669e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:11.017465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9ecce5b2-a617-454f-8247-8c912529669e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:58:11.018709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9ecce5b2-a617-454f-8247-8c912529669e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:58:11.026394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 12:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:58:11.176736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb85259-3729-4b8e-8f0b-6fed26169317 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:58:11.190808 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb85259-3729-4b8e-8f0b-6fed26169317 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:58:11.196052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377db968-98de-4280-baf2-445c9916ae1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:58:11.196684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db968-98de-4280-baf2-445c9916ae1d service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:11.196966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db968-98de-4280-baf2-445c9916ae1d service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:11.197239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db968-98de-4280-baf2-445c9916ae1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:11.205068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377db968-98de-4280-baf2-445c9916ae1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:58:11.206672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377db968-98de-4280-baf2-445c9916ae1d service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:58:11.216459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 762/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:58:15.204499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b0444-3560-4a45-8438-fe51157b8738 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:15.205059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b0444-3560-4a45-8438-fe51157b8738 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:15.205059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b0444-3560-4a45-8438-fe51157b8738 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:15.206371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:58:15.211826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b9478-a6ab-4319-86e4-7a0491ec8bcf None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:15.212237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b9478-a6ab-4319-86e4-7a0491ec8bcf None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:15.212381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b9478-a6ab-4319-86e4-7a0491ec8bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:15.232495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-884b9478-a6ab-4319-86e4-7a0491ec8bcf None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:58:15.233708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-884b9478-a6ab-4319-86e4-7a0491ec8bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:58:15.242205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:58:15.278758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b0444-3560-4a45-8438-fe51157b8738 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:58:15.280444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e74b0444-3560-4a45-8438-fe51157b8738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:58:15.336603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b0444-3560-4a45-8438-fe51157b8738 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:58:15.340149 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc9452-d724-416d-b628-62fc2e21ca44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:58:15.340814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc9452-d724-416d-b628-62fc2e21ca44 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:15.341097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc9452-d724-416d-b628-62fc2e21ca44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:15.341376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc9452-d724-416d-b628-62fc2e21ca44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:15.350331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc9452-d724-416d-b628-62fc2e21ca44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:58:15.351791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc9452-d724-416d-b628-62fc2e21ca44 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:58:15.357093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:58:15.364079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 767/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:58:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:58:17.547964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac5f4d-058b-4f4b-9cfa-b933126afa10 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:17.548521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac5f4d-058b-4f4b-9cfa-b933126afa10 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:17.548521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac5f4d-058b-4f4b-9cfa-b933126afa10 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:17.549912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:58:17.555233 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859dc8a-cb75-4281-bb03-8dc924fbc5e8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:17.555641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859dc8a-cb75-4281-bb03-8dc924fbc5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:17.555803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859dc8a-cb75-4281-bb03-8dc924fbc5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:17.573945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f859dc8a-cb75-4281-bb03-8dc924fbc5e8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:58:17.575168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f859dc8a-cb75-4281-bb03-8dc924fbc5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:58:17.582894 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:58:17.618148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac5f4d-058b-4f4b-9cfa-b933126afa10 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:58:17.629524 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac5f4d-058b-4f4b-9cfa-b933126afa10 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:58:17.633078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57be38-95a3-4be8-8c8e-b66274a6b4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:58:17.633709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57be38-95a3-4be8-8c8e-b66274a6b4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:17.633997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57be38-95a3-4be8-8c8e-b66274a6b4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:17.634272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57be38-95a3-4be8-8c8e-b66274a6b4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:17.642975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57be38-95a3-4be8-8c8e-b66274a6b4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:58:17.644387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57be38-95a3-4be8-8c8e-b66274a6b4c3 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:58:17.653593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 765/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:58:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:58:21.394700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4577b6-e56e-41ca-b74e-a98ba6d648b8 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:21.395302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4577b6-e56e-41ca-b74e-a98ba6d648b8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:21.395302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4577b6-e56e-41ca-b74e-a98ba6d648b8 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:21.396620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:58:21.401850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802cabc-6434-4233-bec2-bdb0395a43dc None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:21.402265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802cabc-6434-4233-bec2-bdb0395a43dc None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:21.402444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802cabc-6434-4233-bec2-bdb0395a43dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:21.420434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e802cabc-6434-4233-bec2-bdb0395a43dc None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:58:21.421620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e802cabc-6434-4233-bec2-bdb0395a43dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:58:21.429280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:58:21.700980 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4577b6-e56e-41ca-b74e-a98ba6d648b8 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:58:21.702706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5a4577b6-e56e-41ca-b74e-a98ba6d648b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:58:21.758497 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4577b6-e56e-41ca-b74e-a98ba6d648b8 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:58:21.761916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47eb24d-086c-4bde-ada7-e7cd2621acc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:58:21.763164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eb24d-086c-4bde-ada7-e7cd2621acc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:21.763443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eb24d-086c-4bde-ada7-e7cd2621acc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:21.763716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eb24d-086c-4bde-ada7-e7cd2621acc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:21.772319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47eb24d-086c-4bde-ada7-e7cd2621acc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:58:21.773743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47eb24d-086c-4bde-ada7-e7cd2621acc5 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:58:21.782911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 770/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:58:24.007239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e89b7-7c82-4c0e-b354-a8591a59814f None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:24.007790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e89b7-7c82-4c0e-b354-a8591a59814f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:24.007790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e89b7-7c82-4c0e-b354-a8591a59814f None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:24.009153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 12:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:58:24.015040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d508c8b-8581-4c44-af07-e499b8ea54d4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:24.015463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d508c8b-8581-4c44-af07-e499b8ea54d4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:24.015606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d508c8b-8581-4c44-af07-e499b8ea54d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:24.033469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-4d508c8b-8581-4c44-af07-e499b8ea54d4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:58:24.034696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4d508c8b-8581-4c44-af07-e499b8ea54d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:58:24.042401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 12:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:58:24.315619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e89b7-7c82-4c0e-b354-a8591a59814f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:58:24.317332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f33e89b7-7c82-4c0e-b354-a8591a59814f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:58:24.330520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e89b7-7c82-4c0e-b354-a8591a59814f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:58:24.333913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9481497-b943-4dec-a02f-016e54d1d95f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:58:24.334586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9481497-b943-4dec-a02f-016e54d1d95f service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:58:24.334870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9481497-b943-4dec-a02f-016e54d1d95f service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:58:24.335146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9481497-b943-4dec-a02f-016e54d1d95f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:58:24.344254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9481497-b943-4dec-a02f-016e54d1d95f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:58:24.345649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9481497-b943-4dec-a02f-016e54d1d95f service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:58:24.354811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 768/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:58:34.028224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=32 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:58:34.350859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=70 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:53.824058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04252365-0272-46fd-988b-c9cc8eef61b3 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:53.824688 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04252365-0272-46fd-988b-c9cc8eef61b3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:53.824688 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04252365-0272-46fd-988b-c9cc8eef61b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:53.832864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27576f-5e4b-4b44-b54c-812d616c4788 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:53.834019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27576f-5e4b-4b44-b54c-812d616c4788 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:53.834594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27576f-5e4b-4b44-b54c-812d616c4788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:53.871349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-04252365-0272-46fd-988b-c9cc8eef61b3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:53.872583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-04252365-0272-46fd-988b-c9cc8eef61b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:53.873680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1a27576f-5e4b-4b44-b54c-812d616c4788 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:53.874949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1a27576f-5e4b-4b44-b54c-812d616c4788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:53.890343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 772/1540] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:53.891992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 769/1541] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 12:59:53.901916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2fb1f-b461-4b14-bcf1-f37adf1f5636 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:53.902230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2fb1f-b461-4b14-bcf1-f37adf1f5636 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:53.902507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2fb1f-b461-4b14-bcf1-f37adf1f5636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:53.910847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78e5da-8c2a-4962-a665-68d2bc1cbee8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:53.911154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78e5da-8c2a-4962-a665-68d2bc1cbee8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:53.911411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78e5da-8c2a-4962-a665-68d2bc1cbee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:53.940902 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-fbf2fb1f-b461-4b14-bcf1-f37adf1f5636 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:53.942045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-fbf2fb1f-b461-4b14-bcf1-f37adf1f5636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:53.943994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-db78e5da-8c2a-4962-a665-68d2bc1cbee8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:53.945624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-db78e5da-8c2a-4962-a665-68d2bc1cbee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:53.960368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 773/1542] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:53.962318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9996998-8113-4aad-b7f8-427dcba8c0ef None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:53.962318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9996998-8113-4aad-b7f8-427dcba8c0ef None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:53.962318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9996998-8113-4aad-b7f8-427dcba8c0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:53.965201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 770/1543] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 12:59:53.969897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78e5da-8c2a-4962-a665-68d2bc1cbee8 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:53.985590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78e5da-8c2a-4962-a665-68d2bc1cbee8 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:53.999652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa561d09-d435-468d-b771-f496e9f4dd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.000363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa561d09-d435-468d-b771-f496e9f4dd56 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.000653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa561d09-d435-468d-b771-f496e9f4dd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.000934 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa561d09-d435-468d-b771-f496e9f4dd56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.001691 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa561d09-d435-468d-b771-f496e9f4dd56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:59:54.002003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa561d09-d435-468d-b771-f496e9f4dd56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.003092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: WARNING py.warnings [None req-fa561d09-d435-468d-b771-f496e9f4dd56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 12:59:54.003092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: warnings.warn(msg) May 24 12:59:54.003092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: May 24 12:59:54.004007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa561d09-d435-468d-b771-f496e9f4dd56 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.010336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b9996998-8113-4aad-b7f8-427dcba8c0ef None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.011659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b9996998-8113-4aad-b7f8-427dcba8c0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.031293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 774/1544] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.032156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9996998-8113-4aad-b7f8-427dcba8c0ef None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.039328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 771/1545] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 12:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:59:54.040875 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149db72b-4198-42f7-accb-ab4adfa4bedd None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.041173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149db72b-4198-42f7-accb-ab4adfa4bedd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.041458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149db72b-4198-42f7-accb-ab4adfa4bedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.052364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9996998-8113-4aad-b7f8-427dcba8c0ef None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.068347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb6074-9d6f-444b-b335-32bda489bb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.069040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb6074-9d6f-444b-b335-32bda489bb1a admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.069330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb6074-9d6f-444b-b335-32bda489bb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.069611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb6074-9d6f-444b-b335-32bda489bb1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.070379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb6074-9d6f-444b-b335-32bda489bb1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:59:54.070694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb6074-9d6f-444b-b335-32bda489bb1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.072475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-149db72b-4198-42f7-accb-ab4adfa4bedd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.072765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb6074-9d6f-444b-b335-32bda489bb1a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.074253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-149db72b-4198-42f7-accb-ab4adfa4bedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.086343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 772/1546] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.087211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149db72b-4198-42f7-accb-ab4adfa4bedd None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.095626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 775/1547] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:59:54.096499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb6074-9d6f-444b-b335-32bda489bb1a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.105274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149db72b-4198-42f7-accb-ab4adfa4bedd None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.115618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bbd69-837e-486f-a1e5-f859bb8b0d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.116332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbd69-837e-486f-a1e5-f859bb8b0d75 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.116619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbd69-837e-486f-a1e5-f859bb8b0d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.116899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbd69-837e-486f-a1e5-f859bb8b0d75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.117626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bbd69-837e-486f-a1e5-f859bb8b0d75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.119679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bbd69-837e-486f-a1e5-f859bb8b0d75 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.120448 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb6074-9d6f-444b-b335-32bda489bb1a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.126232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83583b-084f-43e3-b7bd-240772f9954f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.126967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83583b-084f-43e3-b7bd-240772f9954f admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.127258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83583b-084f-43e3-b7bd-240772f9954f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.127547 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83583b-084f-43e3-b7bd-240772f9954f admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.128314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83583b-084f-43e3-b7bd-240772f9954f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:59:54.128632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83583b-084f-43e3-b7bd-240772f9954f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.130393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83583b-084f-43e3-b7bd-240772f9954f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.160991 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 773/1548] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:54.162079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 776/1549] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:59:54.162518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9a7e-e0e2-466e-b5f6-a6d1695c4a4c None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.162814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9a7e-e0e2-466e-b5f6-a6d1695c4a4c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.163149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9a7e-e0e2-466e-b5f6-a6d1695c4a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.163401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eacec6-9604-4931-a934-0bca5358ae85 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.163697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eacec6-9604-4931-a934-0bca5358ae85 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.163983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eacec6-9604-4931-a934-0bca5358ae85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.203893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c7eacec6-9604-4931-a934-0bca5358ae85 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.208454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c7eacec6-9604-4931-a934-0bca5358ae85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.212446 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-6d6a9a7e-e0e2-466e-b5f6-a6d1695c4a4c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.216061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6d6a9a7e-e0e2-466e-b5f6-a6d1695c4a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.293689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 777/1550] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.294703 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eacec6-9604-4931-a934-0bca5358ae85 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.303702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eacec6-9604-4931-a934-0bca5358ae85 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.308288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180d0e5-5ee0-4f44-bc5c-9e4df7f03c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.308936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d0e5-5ee0-4f44-bc5c-9e4df7f03c9b admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.309052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 774/1551] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.309870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a9a7e-e0e2-466e-b5f6-a6d1695c4a4c None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.312870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d0e5-5ee0-4f44-bc5c-9e4df7f03c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.313200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d0e5-5ee0-4f44-bc5c-9e4df7f03c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.314066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180d0e5-5ee0-4f44-bc5c-9e4df7f03c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.315349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: WARNING py.warnings [None req-9180d0e5-5ee0-4f44-bc5c-9e4df7f03c9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 12:59:54.315349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: warnings.warn(msg) May 24 12:59:54.315349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: May 24 12:59:54.317179 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180d0e5-5ee0-4f44-bc5c-9e4df7f03c9b admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.319413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a9a7e-e0e2-466e-b5f6-a6d1695c4a4c None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.324751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1971c2-ec67-4ea9-a2a3-d5a57e1b116f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.325419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1971c2-ec67-4ea9-a2a3-d5a57e1b116f admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.325701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1971c2-ec67-4ea9-a2a3-d5a57e1b116f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.325981 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1971c2-ec67-4ea9-a2a3-d5a57e1b116f admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.326710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1971c2-ec67-4ea9-a2a3-d5a57e1b116f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.327713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: WARNING py.warnings [None req-dc1971c2-ec67-4ea9-a2a3-d5a57e1b116f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 12:59:54.327713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: warnings.warn(msg) May 24 12:59:54.327713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: May 24 12:59:54.329023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1971c2-ec67-4ea9-a2a3-d5a57e1b116f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.348664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 778/1552] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:54.348664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180d0e5-5ee0-4f44-bc5c-9e4df7f03c9b admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.356204 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 775/1553] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:54.356204 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767ae1b-7536-46fb-9bf4-272e8a5d7180 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.356204 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767ae1b-7536-46fb-9bf4-272e8a5d7180 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.356204 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767ae1b-7536-46fb-9bf4-272e8a5d7180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.373201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1767ae1b-7536-46fb-9bf4-272e8a5d7180 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.374696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1767ae1b-7536-46fb-9bf4-272e8a5d7180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.412445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180d0e5-5ee0-4f44-bc5c-9e4df7f03c9b admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.417145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d309cf-c509-46ca-9c7f-4b94814d90fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.417824 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d309cf-c509-46ca-9c7f-4b94814d90fb admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.418091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d309cf-c509-46ca-9c7f-4b94814d90fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.418404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d309cf-c509-46ca-9c7f-4b94814d90fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.419092 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d309cf-c509-46ca-9c7f-4b94814d90fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.421284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d309cf-c509-46ca-9c7f-4b94814d90fb admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.435238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 776/1554] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.443872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79780b-4466-40a8-b1ab-17a5271a67c8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.443872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79780b-4466-40a8-b1ab-17a5271a67c8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.443872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79780b-4466-40a8-b1ab-17a5271a67c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.455360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-fd79780b-4466-40a8-b1ab-17a5271a67c8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.456114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 779/1555] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:54.456670 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-fd79780b-4466-40a8-b1ab-17a5271a67c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.457615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3682117-0251-4890-bb8a-74e1fea3749f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.457926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3682117-0251-4890-bb8a-74e1fea3749f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.458276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3682117-0251-4890-bb8a-74e1fea3749f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.476181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c3682117-0251-4890-bb8a-74e1fea3749f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.477364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c3682117-0251-4890-bb8a-74e1fea3749f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.482042 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:54.490825 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:54.518611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 777/1556] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.520013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f11f5-b4fd-4973-979e-7112e93264be None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.520407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f11f5-b4fd-4973-979e-7112e93264be None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.520789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f11f5-b4fd-4973-979e-7112e93264be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.539690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a63f11f5-b4fd-4973-979e-7112e93264be None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.540900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a63f11f5-b4fd-4973-979e-7112e93264be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.541345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 780/1557] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.543886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfffeef-9876-4044-b829-f93278962aec None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.544352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfffeef-9876-4044-b829-f93278962aec None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.544797 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfffeef-9876-4044-b829-f93278962aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.549378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 778/1558] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.551124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f11f5-b4fd-4973-979e-7112e93264be None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.561234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f11f5-b4fd-4973-979e-7112e93264be None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.564591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-5cfffeef-9876-4044-b829-f93278962aec None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.565792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5cfffeef-9876-4044-b829-f93278962aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.566131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc92474-2930-49ba-8e3b-f7e39b091b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.566897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc92474-2930-49ba-8e3b-f7e39b091b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.567257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc92474-2930-49ba-8e3b-f7e39b091b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.567625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc92474-2930-49ba-8e3b-f7e39b091b20 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.568499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc92474-2930-49ba-8e3b-f7e39b091b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.570546 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc92474-2930-49ba-8e3b-f7e39b091b20 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.574220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 781/1559] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.575018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfffeef-9876-4044-b829-f93278962aec None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.584886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfffeef-9876-4044-b829-f93278962aec None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.589769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d2aa6-5472-407d-b952-b05c3ad17d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.590475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d2aa6-5472-407d-b952-b05c3ad17d81 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.591226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d2aa6-5472-407d-b952-b05c3ad17d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.591509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d2aa6-5472-407d-b952-b05c3ad17d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.594986 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 779/1560] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/users => generated 286 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:54.595157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d2aa6-5472-407d-b952-b05c3ad17d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.595265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8bc47-298a-46d4-bf7b-07c317c56b1f None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.595265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8bc47-298a-46d4-bf7b-07c317c56b1f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.595466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d2aa6-5472-407d-b952-b05c3ad17d81 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.595565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8bc47-298a-46d4-bf7b-07c317c56b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.607704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 782/1561] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:54.607704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d2aa6-5472-407d-b952-b05c3ad17d81 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.614789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-acf8bc47-298a-46d4-bf7b-07c317c56b1f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.615582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d2aa6-5472-407d-b952-b05c3ad17d81 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.615992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-acf8bc47-298a-46d4-bf7b-07c317c56b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.621268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec596fc5-1350-4e3b-9ef7-fa76c92ce6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.621971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec596fc5-1350-4e3b-9ef7-fa76c92ce6dd admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.622319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec596fc5-1350-4e3b-9ef7-fa76c92ce6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.622597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec596fc5-1350-4e3b-9ef7-fa76c92ce6dd admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.623562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec596fc5-1350-4e3b-9ef7-fa76c92ce6dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.628539 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 780/1562] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.628677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec596fc5-1350-4e3b-9ef7-fa76c92ce6dd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.628789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c656b2-b197-41d4-935c-b6e703ab95a2 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.628789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c656b2-b197-41d4-935c-b6e703ab95a2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.628789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c656b2-b197-41d4-935c-b6e703ab95a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.645894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-28c656b2-b197-41d4-935c-b6e703ab95a2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.649113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 783/1563] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/users => generated 286 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:54.649241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-28c656b2-b197-41d4-935c-b6e703ab95a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.649361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a12f1-757f-4a1a-b7d0-c93dd2dc19a1 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.649361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a12f1-757f-4a1a-b7d0-c93dd2dc19a1 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.649361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a12f1-757f-4a1a-b7d0-c93dd2dc19a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.655527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 781/1564] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.655527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c656b2-b197-41d4-935c-b6e703ab95a2 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.665580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c656b2-b197-41d4-935c-b6e703ab95a2 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.667285 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-de0a12f1-757f-4a1a-b7d0-c93dd2dc19a1 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:54.668511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-de0a12f1-757f-4a1a-b7d0-c93dd2dc19a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:54.670434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752110a-767f-4140-84be-f417787840c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.671178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752110a-767f-4140-84be-f417787840c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.671460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752110a-767f-4140-84be-f417787840c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.671845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752110a-767f-4140-84be-f417787840c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.672570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752110a-767f-4140-84be-f417787840c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:59:54.672880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752110a-767f-4140-84be-f417787840c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.674528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752110a-767f-4140-84be-f417787840c8 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.676444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 784/1565] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:54.677347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a12f1-757f-4a1a-b7d0-c93dd2dc19a1 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.687077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a12f1-757f-4a1a-b7d0-c93dd2dc19a1 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.696345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 782/1566] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:59:54.696345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752110a-767f-4140-84be-f417787840c8 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.698908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e088b-137b-4286-ac62-33b6494f91c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.699568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e088b-137b-4286-ac62-33b6494f91c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.699859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e088b-137b-4286-ac62-33b6494f91c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.699967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752110a-767f-4140-84be-f417787840c8 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.700142 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e088b-137b-4286-ac62-33b6494f91c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.701021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e088b-137b-4286-ac62-33b6494f91c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:59:54.701334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e088b-137b-4286-ac62-33b6494f91c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.702992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e088b-137b-4286-ac62-33b6494f91c2 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.704373 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af337adc-0cae-4ea4-928d-f9e5b116b0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.705004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337adc-0cae-4ea4-928d-f9e5b116b0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.705294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337adc-0cae-4ea4-928d-f9e5b116b0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.705635 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337adc-0cae-4ea4-928d-f9e5b116b0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.706306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af337adc-0cae-4ea4-928d-f9e5b116b0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.707882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af337adc-0cae-4ea4-928d-f9e5b116b0f5 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.722413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 785/1567] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:59:54.722413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e088b-137b-4286-ac62-33b6494f91c2 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.725801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e088b-137b-4286-ac62-33b6494f91c2 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.726048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 783/1568] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:54.726673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af337adc-0cae-4ea4-928d-f9e5b116b0f5 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.733224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4faa4bf-4068-49f5-b2f0-7dd47900171f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.733976 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4faa4bf-4068-49f5-b2f0-7dd47900171f admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.734173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4faa4bf-4068-49f5-b2f0-7dd47900171f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.734487 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4faa4bf-4068-49f5-b2f0-7dd47900171f admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.735215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4faa4bf-4068-49f5-b2f0-7dd47900171f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.736426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af337adc-0cae-4ea4-928d-f9e5b116b0f5 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.736807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4faa4bf-4068-49f5-b2f0-7dd47900171f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.741098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c078c-867b-4883-8971-8697104cda3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.741731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c078c-867b-4883-8971-8697104cda3a admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.742018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c078c-867b-4883-8971-8697104cda3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.742312 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c078c-867b-4883-8971-8697104cda3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.743037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c078c-867b-4883-8971-8697104cda3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:59:54.743347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c078c-867b-4883-8971-8697104cda3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.745558 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c078c-867b-4883-8971-8697104cda3a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.747781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 786/1569] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:54.748821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4faa4bf-4068-49f5-b2f0-7dd47900171f admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.758318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4faa4bf-4068-49f5-b2f0-7dd47900171f admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.761861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 784/1570] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:59:54.762711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c078c-867b-4883-8971-8697104cda3a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.763700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef0949-df70-4ea5-987d-e9363cdca2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.764376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0949-df70-4ea5-987d-e9363cdca2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.764662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0949-df70-4ea5-987d-e9363cdca2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.764942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0949-df70-4ea5-987d-e9363cdca2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.765677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef0949-df70-4ea5-987d-e9363cdca2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.767331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef0949-df70-4ea5-987d-e9363cdca2a2 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.774464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c078c-867b-4883-8971-8697104cda3a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.780748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 787/1571] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:54.780748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef0949-df70-4ea5-987d-e9363cdca2a2 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.781585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fe9b0-4872-418a-badf-bc1470ad7926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.782259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fe9b0-4872-418a-badf-bc1470ad7926 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.782600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fe9b0-4872-418a-badf-bc1470ad7926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.782884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fe9b0-4872-418a-badf-bc1470ad7926 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.783627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fe9b0-4872-418a-badf-bc1470ad7926 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:59:54.783942 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fe9b0-4872-418a-badf-bc1470ad7926 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.785554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fe9b0-4872-418a-badf-bc1470ad7926 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.794602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef0949-df70-4ea5-987d-e9363cdca2a2 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.799448 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00d05e-0be7-493e-a4a3-410c068c6700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.800111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00d05e-0be7-493e-a4a3-410c068c6700 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.800259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 785/1572] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:59:54.800396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00d05e-0be7-493e-a4a3-410c068c6700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.800669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00d05e-0be7-493e-a4a3-410c068c6700 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.801390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00d05e-0be7-493e-a4a3-410c068c6700 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 12:59:54.801703 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00d05e-0be7-493e-a4a3-410c068c6700 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.801824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fe9b0-4872-418a-badf-bc1470ad7926 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.803601 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00d05e-0be7-493e-a4a3-410c068c6700 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.816179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fe9b0-4872-418a-badf-bc1470ad7926 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.820037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 788/1573] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 12:59:54.820888 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00d05e-0be7-493e-a4a3-410c068c6700 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.821026 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6a1d8-5448-43d0-8991-483dbd18d3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.821760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a1d8-5448-43d0-8991-483dbd18d3ac admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.822082 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a1d8-5448-43d0-8991-483dbd18d3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.822458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a1d8-5448-43d0-8991-483dbd18d3ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.823254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6a1d8-5448-43d0-8991-483dbd18d3ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.827012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6a1d8-5448-43d0-8991-483dbd18d3ac admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.831757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00d05e-0be7-493e-a4a3-410c068c6700 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.836877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ee5ae-c96b-440e-a41e-dd155fb03ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.837530 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ee5ae-c96b-440e-a41e-dd155fb03ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.837812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ee5ae-c96b-440e-a41e-dd155fb03ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.838086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ee5ae-c96b-440e-a41e-dd155fb03ef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.840668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ee5ae-c96b-440e-a41e-dd155fb03ef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.843332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ee5ae-c96b-440e-a41e-dd155fb03ef0 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.865978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 786/1574] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:54.866835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6a1d8-5448-43d0-8991-483dbd18d3ac admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.875078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 789/1575] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:54.875742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ee5ae-c96b-440e-a41e-dd155fb03ef0 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.933401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6a1d8-5448-43d0-8991-483dbd18d3ac admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.937727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ee5ae-c96b-440e-a41e-dd155fb03ef0 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.938083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1f0a-1868-4b92-bd30-327a84c424a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.938789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1f0a-1868-4b92-bd30-327a84c424a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.939075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1f0a-1868-4b92-bd30-327a84c424a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.939355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1f0a-1868-4b92-bd30-327a84c424a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.940080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc1f0a-1868-4b92-bd30-327a84c424a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.941678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc1f0a-1868-4b92-bd30-327a84c424a1 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.942407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39053659-6752-4c5d-aff2-12fad8b4e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.943291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39053659-6752-4c5d-aff2-12fad8b4e2c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.943575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39053659-6752-4c5d-aff2-12fad8b4e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.943848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39053659-6752-4c5d-aff2-12fad8b4e2c3 admin admin] PATH_INFO: `/v3/projects/cbc76452e79c4411b4592c239245cf5d/users/39adb89399764ce0af29ceea422f8fe5/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.951392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 787/1576] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:54.951392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1f0a-1868-4b92-bd30-327a84c424a1 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.955070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:54.960016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39053659-6752-4c5d-aff2-12fad8b4e2c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc76452e79c4411b4592c239245cf5d, user_id=39adb89399764ce0af29ceea422f8fe5, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.961288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: WARNING py.warnings [None req-39053659-6752-4c5d-aff2-12fad8b4e2c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 12:59:54.961288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: warnings.warn(msg) May 24 12:59:54.961288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: May 24 12:59:54.962353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1f0a-1868-4b92-bd30-327a84c424a1 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:54.963570 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39053659-6752-4c5d-aff2-12fad8b4e2c3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.966871 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44149826-ec62-484f-85fa-3d2304400a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:54.967513 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44149826-ec62-484f-85fa-3d2304400a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:54.967839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44149826-ec62-484f-85fa-3d2304400a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:54.968121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44149826-ec62-484f-85fa-3d2304400a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:54.968852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44149826-ec62-484f-85fa-3d2304400a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:54.970937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44149826-ec62-484f-85fa-3d2304400a2a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:54.977132 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 790/1577] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:54 2022] PUT /identity/v3/projects/cbc76452e79c4411b4592c239245cf5d/users/39adb89399764ce0af29ceea422f8fe5/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:54.977918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39053659-6752-4c5d-aff2-12fad8b4e2c3 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:54.986791 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:55.001774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 788/1578] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 12:59:54 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:55.005633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44149826-ec62-484f-85fa-3d2304400a2a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.042501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39053659-6752-4c5d-aff2-12fad8b4e2c3 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.066891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44149826-ec62-484f-85fa-3d2304400a2a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.071822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c73783-3252-403b-89b9-bee7c3f77ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.072159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c73783-3252-403b-89b9-bee7c3f77ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.072516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c73783-3252-403b-89b9-bee7c3f77ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.072717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c73783-3252-403b-89b9-bee7c3f77ab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.073413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c73783-3252-403b-89b9-bee7c3f77ab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.075376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c73783-3252-403b-89b9-bee7c3f77ab9 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.098368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245947d9-ad37-4c76-97c0-0c68df762f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.099047 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245947d9-ad37-4c76-97c0-0c68df762f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.099327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245947d9-ad37-4c76-97c0-0c68df762f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.099608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245947d9-ad37-4c76-97c0-0c68df762f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.103169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 789/1579] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/projects => generated 391 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:55.103297 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245947d9-ad37-4c76-97c0-0c68df762f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.103412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c73783-3252-403b-89b9-bee7c3f77ab9 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.103493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245947d9-ad37-4c76-97c0-0c68df762f50 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.112708 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 791/1580] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:55.113399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245947d9-ad37-4c76-97c0-0c68df762f50 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.126862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245947d9-ad37-4c76-97c0-0c68df762f50 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.162990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c73783-3252-403b-89b9-bee7c3f77ab9 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.168176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b6f78-c953-4a69-9112-a1c6135d31e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.168840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b6f78-c953-4a69-9112-a1c6135d31e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.169135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b6f78-c953-4a69-9112-a1c6135d31e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.169416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b6f78-c953-4a69-9112-a1c6135d31e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.170143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b6f78-c953-4a69-9112-a1c6135d31e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.172094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b6f78-c953-4a69-9112-a1c6135d31e6 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.181839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c5623-560f-49b4-85ee-201b0712f527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.182536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c5623-560f-49b4-85ee-201b0712f527 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.182817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c5623-560f-49b4-85ee-201b0712f527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.183093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c5623-560f-49b4-85ee-201b0712f527 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.183772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c5623-560f-49b4-85ee-201b0712f527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.185704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c5623-560f-49b4-85ee-201b0712f527 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.197725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 790/1581] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:55.198441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b6f78-c953-4a69-9112-a1c6135d31e6 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.206310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 792/1582] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:55.207198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c5623-560f-49b4-85ee-201b0712f527 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.218909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c5623-560f-49b4-85ee-201b0712f527 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.238650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:55.259828 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b6f78-c953-4a69-9112-a1c6135d31e6 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.265019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2012121-e533-41d8-8fa7-9727dffbe69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.265685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2012121-e533-41d8-8fa7-9727dffbe69f admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.265971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2012121-e533-41d8-8fa7-9727dffbe69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.266244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2012121-e533-41d8-8fa7-9727dffbe69f admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.266969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2012121-e533-41d8-8fa7-9727dffbe69f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.268869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2012121-e533-41d8-8fa7-9727dffbe69f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.277326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:55.278945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e09bda-49ec-4f4a-8adc-501d0443b51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.279985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e09bda-49ec-4f4a-8adc-501d0443b51d admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.280399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e09bda-49ec-4f4a-8adc-501d0443b51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.280709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e09bda-49ec-4f4a-8adc-501d0443b51d admin admin] PATH_INFO: `/v3/projects/30d674774c41482ab88fb61053f56c35/users/ef208b9fd08940a180e3468ecd4e821f/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.289824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 791/1583] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:55.291186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcb0ac-b36e-4798-a6fc-c7240fe44497 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.291485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcb0ac-b36e-4798-a6fc-c7240fe44497 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.291912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcb0ac-b36e-4798-a6fc-c7240fe44497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.297036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e09bda-49ec-4f4a-8adc-501d0443b51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=30d674774c41482ab88fb61053f56c35, user_id=ef208b9fd08940a180e3468ecd4e821f, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.299837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e09bda-49ec-4f4a-8adc-501d0443b51d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.309596 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e0bcb0ac-b36e-4798-a6fc-c7240fe44497 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:55.310883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e0bcb0ac-b36e-4798-a6fc-c7240fe44497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:55.314033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 793/1584] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:55 2022] PUT /identity/v3/projects/30d674774c41482ab88fb61053f56c35/users/ef208b9fd08940a180e3468ecd4e821f/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:55.319183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e09bda-49ec-4f4a-8adc-501d0443b51d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.370276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 792/1585] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:55.371229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcb0ac-b36e-4798-a6fc-c7240fe44497 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.380611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcb0ac-b36e-4798-a6fc-c7240fe44497 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.388615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f777aaa-ec12-4b52-8d01-75a49cceca2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.389296 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f777aaa-ec12-4b52-8d01-75a49cceca2f admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.389609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f777aaa-ec12-4b52-8d01-75a49cceca2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.389888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f777aaa-ec12-4b52-8d01-75a49cceca2f admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.390613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f777aaa-ec12-4b52-8d01-75a49cceca2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.391897 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e09bda-49ec-4f4a-8adc-501d0443b51d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.392733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f777aaa-ec12-4b52-8d01-75a49cceca2f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.396693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf61fb7-2ba1-4b52-ae60-cc88d061cd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.397394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf61fb7-2ba1-4b52-ae60-cc88d061cd45 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.397681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf61fb7-2ba1-4b52-ae60-cc88d061cd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.397966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf61fb7-2ba1-4b52-ae60-cc88d061cd45 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.398728 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf61fb7-2ba1-4b52-ae60-cc88d061cd45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.400663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf61fb7-2ba1-4b52-ae60-cc88d061cd45 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.414768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 793/1586] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:55.415681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f777aaa-ec12-4b52-8d01-75a49cceca2f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.425037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 794/1587] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:55.425502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f777aaa-ec12-4b52-8d01-75a49cceca2f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.425848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf61fb7-2ba1-4b52-ae60-cc88d061cd45 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.430823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3718241-4455-4add-83eb-d1fd55dca75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.432826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3718241-4455-4add-83eb-d1fd55dca75a admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.433131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3718241-4455-4add-83eb-d1fd55dca75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.433422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3718241-4455-4add-83eb-d1fd55dca75a admin admin] PATH_INFO: `/v3/projects/ec3ca507ef134b528c0469eee600b0c6/users/65040649551741d5ad79ead288213a32/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.436034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf61fb7-2ba1-4b52-ae60-cc88d061cd45 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.442858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597bb3-888f-4450-bf33-23a9183f26bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.443532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597bb3-888f-4450-bf33-23a9183f26bc admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.443812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597bb3-888f-4450-bf33-23a9183f26bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.444113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597bb3-888f-4450-bf33-23a9183f26bc admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.444800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21597bb3-888f-4450-bf33-23a9183f26bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.446762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21597bb3-888f-4450-bf33-23a9183f26bc admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.450878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3718241-4455-4add-83eb-d1fd55dca75a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec3ca507ef134b528c0469eee600b0c6, user_id=65040649551741d5ad79ead288213a32, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.453528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3718241-4455-4add-83eb-d1fd55dca75a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.467305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 794/1588] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:55 2022] PUT /identity/v3/projects/ec3ca507ef134b528c0469eee600b0c6/users/65040649551741d5ad79ead288213a32/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:55.468225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3718241-4455-4add-83eb-d1fd55dca75a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.472014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 795/1589] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:55.472014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597bb3-888f-4450-bf33-23a9183f26bc admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.485413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597bb3-888f-4450-bf33-23a9183f26bc admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.534732 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3718241-4455-4add-83eb-d1fd55dca75a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.539641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293b146-1f6e-4743-9fc4-66084eb954b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.540315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b146-1f6e-4743-9fc4-66084eb954b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.540601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b146-1f6e-4743-9fc4-66084eb954b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.540878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b146-1f6e-4743-9fc4-66084eb954b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.542143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293b146-1f6e-4743-9fc4-66084eb954b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.542264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b435f3a-12cd-45de-a67b-83e2bd81e5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.542862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435f3a-12cd-45de-a67b-83e2bd81e5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.543144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435f3a-12cd-45de-a67b-83e2bd81e5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.543415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435f3a-12cd-45de-a67b-83e2bd81e5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.543773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293b146-1f6e-4743-9fc4-66084eb954b2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.544128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b435f3a-12cd-45de-a67b-83e2bd81e5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.545744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b435f3a-12cd-45de-a67b-83e2bd81e5c0 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.553583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 795/1590] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:55.554907 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9568-0215-4ded-b4f2-5789b64c4566 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.555200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9568-0215-4ded-b4f2-5789b64c4566 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.555392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 796/1591] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:55.555503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9568-0215-4ded-b4f2-5789b64c4566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.556120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b435f3a-12cd-45de-a67b-83e2bd81e5c0 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.558040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5b435f3a-12cd-45de-a67b-83e2bd81e5c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:55.571594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:55.583029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a0fa9568-0215-4ded-b4f2-5789b64c4566 None None] MFA Rules not processed for user `ef208b9fd08940a180e3468ecd4e821f`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:55.584237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a0fa9568-0215-4ded-b4f2-5789b64c4566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:55.614363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b435f3a-12cd-45de-a67b-83e2bd81e5c0 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.618006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66557734-53e3-423c-bc20-00d0c7cfb9f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.618679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557734-53e3-423c-bc20-00d0c7cfb9f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.618959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557734-53e3-423c-bc20-00d0c7cfb9f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.619233 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557734-53e3-423c-bc20-00d0c7cfb9f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.621817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:55.629618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66557734-53e3-423c-bc20-00d0c7cfb9f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.631080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66557734-53e3-423c-bc20-00d0c7cfb9f0 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.641309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 797/1592] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:59:55.642218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66557734-53e3-423c-bc20-00d0c7cfb9f0 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.649310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 796/1593] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:55.650133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa9568-0215-4ded-b4f2-5789b64c4566 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.657118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66557734-53e3-423c-bc20-00d0c7cfb9f0 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.664058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e94d19-b524-4177-a6b9-982e951077dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.665278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e94d19-b524-4177-a6b9-982e951077dd admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.665514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa9568-0215-4ded-b4f2-5789b64c4566 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.666835 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e94d19-b524-4177-a6b9-982e951077dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.667461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e94d19-b524-4177-a6b9-982e951077dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.668295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e94d19-b524-4177-a6b9-982e951077dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.670393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e94d19-b524-4177-a6b9-982e951077dd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.672736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714a19f-437b-4db5-a013-d63e5a208096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.673160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714a19f-437b-4db5-a013-d63e5a208096 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.673507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714a19f-437b-4db5-a013-d63e5a208096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.674019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714a19f-437b-4db5-a013-d63e5a208096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.674931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714a19f-437b-4db5-a013-d63e5a208096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.676662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714a19f-437b-4db5-a013-d63e5a208096 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.687489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 798/1594] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:55.688682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e94d19-b524-4177-a6b9-982e951077dd admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.689407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 797/1595] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:55.693236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f883b8-30f3-4b59-92bd-a7a84f08989f None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.693236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f883b8-30f3-4b59-92bd-a7a84f08989f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.693236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f883b8-30f3-4b59-92bd-a7a84f08989f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.706138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e94d19-b524-4177-a6b9-982e951077dd admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.711242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a498726-cf22-40ab-b51e-eff1ef95871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.714370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a498726-cf22-40ab-b51e-eff1ef95871d admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.714370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a498726-cf22-40ab-b51e-eff1ef95871d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.714370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a498726-cf22-40ab-b51e-eff1ef95871d admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.715109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a498726-cf22-40ab-b51e-eff1ef95871d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.716802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a498726-cf22-40ab-b51e-eff1ef95871d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.725957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a8f883b8-30f3-4b59-92bd-a7a84f08989f None None] MFA Rules not processed for user `65040649551741d5ad79ead288213a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:55.727362 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a8f883b8-30f3-4b59-92bd-a7a84f08989f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:55.729100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 799/1596] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:55.730068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a498726-cf22-40ab-b51e-eff1ef95871d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.743187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a498726-cf22-40ab-b51e-eff1ef95871d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.751345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b57efa-302a-4020-bfe1-6f99bb31d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.752026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57efa-302a-4020-bfe1-6f99bb31d561 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.752313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57efa-302a-4020-bfe1-6f99bb31d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.752586 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57efa-302a-4020-bfe1-6f99bb31d561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.753697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b57efa-302a-4020-bfe1-6f99bb31d561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.756618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b57efa-302a-4020-bfe1-6f99bb31d561 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.768875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 800/1597] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:55.772835 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b57efa-302a-4020-bfe1-6f99bb31d561 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.783614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b57efa-302a-4020-bfe1-6f99bb31d561 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.789615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec305664-8a14-4b9f-93bf-9a94019fa6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.790706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec305664-8a14-4b9f-93bf-9a94019fa6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.790998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec305664-8a14-4b9f-93bf-9a94019fa6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.791286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec305664-8a14-4b9f-93bf-9a94019fa6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.792076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec305664-8a14-4b9f-93bf-9a94019fa6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.793680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec305664-8a14-4b9f-93bf-9a94019fa6c7 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.805200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 801/1598] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:55.808335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37548464-8d78-49c8-92ef-0373e426a1b9 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.808600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 798/1599] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:55.808994 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37548464-8d78-49c8-92ef-0373e426a1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.809486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f883b8-30f3-4b59-92bd-a7a84f08989f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.809660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37548464-8d78-49c8-92ef-0373e426a1b9 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.812192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 802/1600] 10.222.0.45 () {54 vars in 961 bytes} [Tue May 24 12:59:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:59:55.813770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37548464-8d78-49c8-92ef-0373e426a1b9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.821404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f883b8-30f3-4b59-92bd-a7a84f08989f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.827597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121e4ed-cf0f-44d1-a084-be78e640193c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.828438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121e4ed-cf0f-44d1-a084-be78e640193c admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.828982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121e4ed-cf0f-44d1-a084-be78e640193c admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.829385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121e4ed-cf0f-44d1-a084-be78e640193c admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.830561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121e4ed-cf0f-44d1-a084-be78e640193c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.832161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37548464-8d78-49c8-92ef-0373e426a1b9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.832709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121e4ed-cf0f-44d1-a084-be78e640193c admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.838103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304951a6-1736-45e4-a391-3e9ac9d51bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.839833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304951a6-1736-45e4-a391-3e9ac9d51bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.839833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304951a6-1736-45e4-a391-3e9ac9d51bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.840127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304951a6-1736-45e4-a391-3e9ac9d51bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.840937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304951a6-1736-45e4-a391-3e9ac9d51bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.842917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304951a6-1736-45e4-a391-3e9ac9d51bc8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.845731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 799/1601] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:55.846657 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121e4ed-cf0f-44d1-a084-be78e640193c admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.854396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 803/1602] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:55.855468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304951a6-1736-45e4-a391-3e9ac9d51bc8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.857605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121e4ed-cf0f-44d1-a084-be78e640193c admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.863721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2892e-57b3-470f-b39c-783fea8a18e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.863880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2892e-57b3-470f-b39c-783fea8a18e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.864263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2892e-57b3-470f-b39c-783fea8a18e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.864609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2892e-57b3-470f-b39c-783fea8a18e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.865400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2892e-57b3-470f-b39c-783fea8a18e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.866493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304951a6-1736-45e4-a391-3e9ac9d51bc8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.867209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2892e-57b3-470f-b39c-783fea8a18e2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.871738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33219e99-6517-4b54-aec6-fda64cb0b5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.872713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33219e99-6517-4b54-aec6-fda64cb0b5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.873075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33219e99-6517-4b54-aec6-fda64cb0b5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.873681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33219e99-6517-4b54-aec6-fda64cb0b5d5 admin admin] PATH_INFO: `/v3/projects/e57720cd453848a89315ad87ace38161/users/f7b0dfb3d6514817aba4ab34fd972ba1/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.882455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 800/1603] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:55.883454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2892e-57b3-470f-b39c-783fea8a18e2 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.892351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33219e99-6517-4b54-aec6-fda64cb0b5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e57720cd453848a89315ad87ace38161, user_id=f7b0dfb3d6514817aba4ab34fd972ba1, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.893964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2892e-57b3-470f-b39c-783fea8a18e2 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:55.896290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33219e99-6517-4b54-aec6-fda64cb0b5d5 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.899211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ace2e-1b9a-4e5d-82fa-1807c5bd67d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:55.900068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ace2e-1b9a-4e5d-82fa-1807c5bd67d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.900364 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ace2e-1b9a-4e5d-82fa-1807c5bd67d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.900645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ace2e-1b9a-4e5d-82fa-1807c5bd67d8 admin admin] PATH_INFO: `/v3/projects/728de65d013a489594d837589c7b978b/users/cb3d46a7a0ae40e9b157b7199b18de27/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.924149 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 804/1604] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:55 2022] PUT /identity/v3/projects/e57720cd453848a89315ad87ace38161/users/f7b0dfb3d6514817aba4ab34fd972ba1/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:55.926196 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33219e99-6517-4b54-aec6-fda64cb0b5d5 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:55.928316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ace2e-1b9a-4e5d-82fa-1807c5bd67d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728de65d013a489594d837589c7b978b, user_id=cb3d46a7a0ae40e9b157b7199b18de27, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:55.931832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ace2e-1b9a-4e5d-82fa-1807c5bd67d8 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:55.952109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 801/1605] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:55 2022] PUT /identity/v3/projects/728de65d013a489594d837589c7b978b/users/cb3d46a7a0ae40e9b157b7199b18de27/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:55.953501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a3258-bc83-442e-a551-8b400bba3cd9 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:55.953791 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a3258-bc83-442e-a551-8b400bba3cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:55.954069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a3258-bc83-442e-a551-8b400bba3cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:55.977663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-024a3258-bc83-442e-a551-8b400bba3cd9 None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:55.978831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-024a3258-bc83-442e-a551-8b400bba3cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:56.013452 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33219e99-6517-4b54-aec6-fda64cb0b5d5 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:56.062859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:56.066190 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 802/1606] 10.222.0.45 () {58 vars in 1058 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:56.067085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a3258-bc83-442e-a551-8b400bba3cd9 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:56.076869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:56.093141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c1202-d7da-4c7f-801c-57b62f4b6b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:56.094006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1202-d7da-4c7f-801c-57b62f4b6b70 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.094398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1202-d7da-4c7f-801c-57b62f4b6b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.094746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1202-d7da-4c7f-801c-57b62f4b6b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.095692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c1202-d7da-4c7f-801c-57b62f4b6b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:56.097786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c1202-d7da-4c7f-801c-57b62f4b6b70 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:56.131761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 805/1607] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 12:59:55 2022] POST /identity/v3/projects => generated 371 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:56.132803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c1202-d7da-4c7f-801c-57b62f4b6b70 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:56.165943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a3258-bc83-442e-a551-8b400bba3cd9 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:56.209367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c1202-d7da-4c7f-801c-57b62f4b6b70 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:56.215697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de03f7-f598-4bb4-8545-4df3dd32dd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:56.216743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de03f7-f598-4bb4-8545-4df3dd32dd59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.217061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de03f7-f598-4bb4-8545-4df3dd32dd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.217435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de03f7-f598-4bb4-8545-4df3dd32dd59 admin admin] PATH_INFO: `/v3/projects/c9a85b3254d14f45a87f6d58df821d4b/users/de8e37ee2be84839b16bdfc1f55004f9/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.236117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de03f7-f598-4bb4-8545-4df3dd32dd59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a85b3254d14f45a87f6d58df821d4b, user_id=de8e37ee2be84839b16bdfc1f55004f9, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:56.238102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9525ec7-73ba-4c06-b54c-aa633d2a14bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:56.238828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de03f7-f598-4bb4-8545-4df3dd32dd59 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:56.239180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9525ec7-73ba-4c06-b54c-aa633d2a14bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.239467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9525ec7-73ba-4c06-b54c-aa633d2a14bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.239742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9525ec7-73ba-4c06-b54c-aa633d2a14bb admin admin] PATH_INFO: `/v3/projects/a92196ee4d214c999d303c7f8e28a9ce/users/91a76f7af8a5467bbf567ca5b9db9ce0/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.265468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 806/1608] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:56 2022] PUT /identity/v3/projects/c9a85b3254d14f45a87f6d58df821d4b/users/de8e37ee2be84839b16bdfc1f55004f9/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:56.265468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de03f7-f598-4bb4-8545-4df3dd32dd59 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:56.268653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9525ec7-73ba-4c06-b54c-aa633d2a14bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a92196ee4d214c999d303c7f8e28a9ce, user_id=91a76f7af8a5467bbf567ca5b9db9ce0, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:56.271922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9525ec7-73ba-4c06-b54c-aa633d2a14bb admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:56.288745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 803/1609] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:56 2022] PUT /identity/v3/projects/a92196ee4d214c999d303c7f8e28a9ce/users/91a76f7af8a5467bbf567ca5b9db9ce0/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:56.291012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f5714-c28a-4822-a695-52fe07916bd0 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.291313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f5714-c28a-4822-a695-52fe07916bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.291654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f5714-c28a-4822-a695-52fe07916bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.310355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-f45f5714-c28a-4822-a695-52fe07916bd0 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:56.311579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f45f5714-c28a-4822-a695-52fe07916bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:56.331877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de03f7-f598-4bb4-8545-4df3dd32dd59 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:56.381367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 804/1610] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:56.389024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002efb1-b0ee-41e6-bfd0-5c37e2170db6 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.389024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002efb1-b0ee-41e6-bfd0-5c37e2170db6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.389024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002efb1-b0ee-41e6-bfd0-5c37e2170db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.410444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68827355-46b2-40a7-87e9-00c9f2aa101c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:56.411346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68827355-46b2-40a7-87e9-00c9f2aa101c admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.411632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68827355-46b2-40a7-87e9-00c9f2aa101c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.412192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68827355-46b2-40a7-87e9-00c9f2aa101c admin admin] PATH_INFO: `/v3/projects/e2bcca0447ec41b798421f242ec00ce2/users/d54c436b2634460caa5d11a9868e9095/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.414413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:56.416960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-0002efb1-b0ee-41e6-bfd0-5c37e2170db6 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:56.418242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-0002efb1-b0ee-41e6-bfd0-5c37e2170db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:56.427895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 805/1611] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:56.428987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002efb1-b0ee-41e6-bfd0-5c37e2170db6 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:56.429706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68827355-46b2-40a7-87e9-00c9f2aa101c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2bcca0447ec41b798421f242ec00ce2, user_id=d54c436b2634460caa5d11a9868e9095, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:56.432892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68827355-46b2-40a7-87e9-00c9f2aa101c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:56.450818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 807/1612] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:56 2022] PUT /identity/v3/projects/e2bcca0447ec41b798421f242ec00ce2/users/d54c436b2634460caa5d11a9868e9095/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:56.451939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68827355-46b2-40a7-87e9-00c9f2aa101c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:56.471328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:56.536017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002efb1-b0ee-41e6-bfd0-5c37e2170db6 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:56.538506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68827355-46b2-40a7-87e9-00c9f2aa101c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:56.547817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1326260-f047-499d-affc-5c541eb13379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:56.548685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1326260-f047-499d-affc-5c541eb13379 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.549138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1326260-f047-499d-affc-5c541eb13379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.549452 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1326260-f047-499d-affc-5c541eb13379 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.550526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1326260-f047-499d-affc-5c541eb13379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:56.552504 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1326260-f047-499d-affc-5c541eb13379 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:56.586735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 808/1613] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 12:59:56 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:56.588108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4f984-9d3e-4005-9019-540023f93f40 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.588451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4f984-9d3e-4005-9019-540023f93f40 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.589276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4f984-9d3e-4005-9019-540023f93f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.625180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-14f4f984-9d3e-4005-9019-540023f93f40 None None] MFA Rules not processed for user `de8e37ee2be84839b16bdfc1f55004f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:56.625452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64177af9-7312-4b3d-acbb-e64a54edd5e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:56.626699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64177af9-7312-4b3d-acbb-e64a54edd5e5 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.626699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64177af9-7312-4b3d-acbb-e64a54edd5e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.626931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-14f4f984-9d3e-4005-9019-540023f93f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:56.627016 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64177af9-7312-4b3d-acbb-e64a54edd5e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.690774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64177af9-7312-4b3d-acbb-e64a54edd5e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:56.692400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64177af9-7312-4b3d-acbb-e64a54edd5e5 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:56.703255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 806/1614] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:59:56.704610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45d79a-ffd4-4a6a-89ab-a7aa07e2ec28 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.705034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45d79a-ffd4-4a6a-89ab-a7aa07e2ec28 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.705320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45d79a-ffd4-4a6a-89ab-a7aa07e2ec28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.732367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-db45d79a-ffd4-4a6a-89ab-a7aa07e2ec28 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:56.733834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-db45d79a-ffd4-4a6a-89ab-a7aa07e2ec28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:56.734503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 809/1615] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:56.739378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4f984-9d3e-4005-9019-540023f93f40 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:56.744019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 807/1616] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:56.745610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45d79a-ffd4-4a6a-89ab-a7aa07e2ec28 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:56.805384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4f984-9d3e-4005-9019-540023f93f40 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:56.809786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4806d7-257d-49eb-898a-40e55306c750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:56.810687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4806d7-257d-49eb-898a-40e55306c750 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.811075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4806d7-257d-49eb-898a-40e55306c750 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.811458 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4806d7-257d-49eb-898a-40e55306c750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.822006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4806d7-257d-49eb-898a-40e55306c750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:56.824300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4806d7-257d-49eb-898a-40e55306c750 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:56.834273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 810/1617] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:59:56.835816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30194-d4bc-4877-a870-67cb871e29ba None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.836172 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30194-d4bc-4877-a870-67cb871e29ba None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.836528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30194-d4bc-4877-a870-67cb871e29ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.837746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45d79a-ffd4-4a6a-89ab-a7aa07e2ec28 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:56.841418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6237693-80c9-419e-8537-d83833428971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:56.842064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6237693-80c9-419e-8537-d83833428971 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.842380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6237693-80c9-419e-8537-d83833428971 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.842659 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6237693-80c9-419e-8537-d83833428971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.849449 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:56.853044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6237693-80c9-419e-8537-d83833428971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:56.854783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6237693-80c9-419e-8537-d83833428971 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:56.859411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:56.867363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 808/1618] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:59:56.869231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf9cf7-0f14-4eb2-a3ec-3af38de33b16 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:56.869391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf9cf7-0f14-4eb2-a3ec-3af38de33b16 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:56.870587 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf9cf7-0f14-4eb2-a3ec-3af38de33b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:56.878888 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-68d30194-d4bc-4877-a870-67cb871e29ba None None] MFA Rules not processed for user `d54c436b2634460caa5d11a9868e9095`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:56.880322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-68d30194-d4bc-4877-a870-67cb871e29ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:56.900126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9fdf9cf7-0f14-4eb2-a3ec-3af38de33b16 None None] MFA Rules not processed for user `39adb89399764ce0af29ceea422f8fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:56.902326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9fdf9cf7-0f14-4eb2-a3ec-3af38de33b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:56.987987 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 811/1619] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:56.989269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30194-d4bc-4877-a870-67cb871e29ba None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:56.994818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 809/1620] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:56.995874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf9cf7-0f14-4eb2-a3ec-3af38de33b16 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:57.013497 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30194-d4bc-4877-a870-67cb871e29ba None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:57.020576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5931d15-11c4-48ba-9713-3fc79d5529da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:57.021474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931d15-11c4-48ba-9713-3fc79d5529da admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.021766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931d15-11c4-48ba-9713-3fc79d5529da admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.022347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931d15-11c4-48ba-9713-3fc79d5529da admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.022810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5931d15-11c4-48ba-9713-3fc79d5529da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:57.024449 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5931d15-11c4-48ba-9713-3fc79d5529da admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:57.042340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 812/1621] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:57.045791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5931d15-11c4-48ba-9713-3fc79d5529da admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:57.054656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5931d15-11c4-48ba-9713-3fc79d5529da admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:57.059479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff8905-74b9-451b-b6f3-68beef6592eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:57.060622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff8905-74b9-451b-b6f3-68beef6592eb service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.061416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff8905-74b9-451b-b6f3-68beef6592eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.062215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff8905-74b9-451b-b6f3-68beef6592eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.079100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff8905-74b9-451b-b6f3-68beef6592eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:57.081947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff8905-74b9-451b-b6f3-68beef6592eb service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:57.087657 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf9cf7-0f14-4eb2-a3ec-3af38de33b16 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:57.093067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a944341-6bd5-4b5a-bef0-728f74645847 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:57.093731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a944341-6bd5-4b5a-bef0-728f74645847 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.094063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a944341-6bd5-4b5a-bef0-728f74645847 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.094374 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a944341-6bd5-4b5a-bef0-728f74645847 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.104547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a944341-6bd5-4b5a-bef0-728f74645847 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:57.106097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a944341-6bd5-4b5a-bef0-728f74645847 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:57.107526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 813/1622] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:59:57.109115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba6e34-330e-45f4-a1bd-5f02424e2732 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.109356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba6e34-330e-45f4-a1bd-5f02424e2732 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.109639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba6e34-330e-45f4-a1bd-5f02424e2732 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.111925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 814/1623] 10.222.0.45 () {54 vars in 912 bytes} [Tue May 24 12:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:59:57.113143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba6e34-330e-45f4-a1bd-5f02424e2732 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:57.121373 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 810/1624] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:59:57.124193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38e12-44ae-4767-b8d6-cd56ded6128c None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.124841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38e12-44ae-4767-b8d6-cd56ded6128c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.124961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38e12-44ae-4767-b8d6-cd56ded6128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.131476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba6e34-330e-45f4-a1bd-5f02424e2732 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:57.138521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f1f71-e912-41f8-8631-6e7c6a4e4f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:57.139198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f1f71-e912-41f8-8631-6e7c6a4e4f94 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.139486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f1f71-e912-41f8-8631-6e7c6a4e4f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.139826 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f1f71-e912-41f8-8631-6e7c6a4e4f94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.141106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f1f71-e912-41f8-8631-6e7c6a4e4f94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:57.143215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f1f71-e912-41f8-8631-6e7c6a4e4f94 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:57.161485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1fd38e12-44ae-4767-b8d6-cd56ded6128c None None] MFA Rules not processed for user `39adb89399764ce0af29ceea422f8fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:57.163632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1fd38e12-44ae-4767-b8d6-cd56ded6128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:57.180201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 811/1625] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:57.180201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd38e12-44ae-4767-b8d6-cd56ded6128c None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:57.194186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 815/1626] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 12:59:57 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:57.195582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b497099-6d0d-450a-9220-8dde3bb6454f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.195877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b497099-6d0d-450a-9220-8dde3bb6454f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.196157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b497099-6d0d-450a-9220-8dde3bb6454f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.204273 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd38e12-44ae-4767-b8d6-cd56ded6128c None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:57.235743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-3b497099-6d0d-450a-9220-8dde3bb6454f None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:57.240091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3b497099-6d0d-450a-9220-8dde3bb6454f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:57.298938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132841e5-9880-46ea-950d-206b75cd3bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:57.300575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132841e5-9880-46ea-950d-206b75cd3bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.301798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132841e5-9880-46ea-950d-206b75cd3bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.301798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132841e5-9880-46ea-950d-206b75cd3bb6 admin admin] PATH_INFO: `/v3/projects/19d87d484a9f460e873612d2d77e54a8/users/07d1a207bc104128b123fa23a4493701/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.377216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132841e5-9880-46ea-950d-206b75cd3bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d87d484a9f460e873612d2d77e54a8, user_id=07d1a207bc104128b123fa23a4493701, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:57.386081 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132841e5-9880-46ea-950d-206b75cd3bb6 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:57.417163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 816/1627] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 12:59:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:57.423909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b497099-6d0d-450a-9220-8dde3bb6454f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:57.427456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 812/1628] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:57 2022] PUT /identity/v3/projects/19d87d484a9f460e873612d2d77e54a8/users/07d1a207bc104128b123fa23a4493701/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:57.432835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a316a-079f-454a-878e-ab8d55a19aaf None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.436215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a316a-079f-454a-878e-ab8d55a19aaf None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.437355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a316a-079f-454a-878e-ab8d55a19aaf None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.442169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 813/1629] 10.222.0.45 () {54 vars in 961 bytes} [Tue May 24 12:59:57 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:59:57.444487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a316a-079f-454a-878e-ab8d55a19aaf None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:57.451072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b497099-6d0d-450a-9220-8dde3bb6454f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:57.469175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:57.543970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:57.586187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d15a2a-dfc3-4f29-aab6-cf4cce107075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:57.587567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15a2a-dfc3-4f29-aab6-cf4cce107075 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.588129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15a2a-dfc3-4f29-aab6-cf4cce107075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.588552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15a2a-dfc3-4f29-aab6-cf4cce107075 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.589499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d15a2a-dfc3-4f29-aab6-cf4cce107075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:57.590111 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a316a-079f-454a-878e-ab8d55a19aaf None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:57.591996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d15a2a-dfc3-4f29-aab6-cf4cce107075 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:57.597752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc317d-070f-44c9-aee7-5d3a10bd5156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:57.599184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc317d-070f-44c9-aee7-5d3a10bd5156 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.600182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc317d-070f-44c9-aee7-5d3a10bd5156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.600752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc317d-070f-44c9-aee7-5d3a10bd5156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.601694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc317d-070f-44c9-aee7-5d3a10bd5156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:57.603668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc317d-070f-44c9-aee7-5d3a10bd5156 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:57.616976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 814/1630] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:57.618094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc317d-070f-44c9-aee7-5d3a10bd5156 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:57.626703 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 817/1631] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 12:59:57 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 12:59:57.628251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec493f6-9410-4c44-b19f-87f240452a6c None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.628606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec493f6-9410-4c44-b19f-87f240452a6c None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.628881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec493f6-9410-4c44-b19f-87f240452a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.655330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7ec493f6-9410-4c44-b19f-87f240452a6c None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:57.656972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7ec493f6-9410-4c44-b19f-87f240452a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:57.695679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc317d-070f-44c9-aee7-5d3a10bd5156 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:57.700509 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb6754-0cf0-49b1-8e1b-3e641d73e724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:57.701607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb6754-0cf0-49b1-8e1b-3e641d73e724 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.702009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb6754-0cf0-49b1-8e1b-3e641d73e724 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.702444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb6754-0cf0-49b1-8e1b-3e641d73e724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.805970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 818/1632] 10.222.0.45 () {58 vars in 1058 bytes} [Tue May 24 12:59:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:57.807378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec493f6-9410-4c44-b19f-87f240452a6c None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:57.824154 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec493f6-9410-4c44-b19f-87f240452a6c None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:57.832340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd13c1d-519b-4cdd-9d32-f3c46bb21784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:57.833364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd13c1d-519b-4cdd-9d32-f3c46bb21784 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.833984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd13c1d-519b-4cdd-9d32-f3c46bb21784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.834274 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd13c1d-519b-4cdd-9d32-f3c46bb21784 admin admin] PATH_INFO: `/v3/projects/19d87d484a9f460e873612d2d77e54a8/users/07d1a207bc104128b123fa23a4493701/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.837018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd13c1d-519b-4cdd-9d32-f3c46bb21784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d87d484a9f460e873612d2d77e54a8, user_id=07d1a207bc104128b123fa23a4493701, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:57.841374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd13c1d-519b-4cdd-9d32-f3c46bb21784 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:57.868613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb6754-0cf0-49b1-8e1b-3e641d73e724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:57.870110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb6754-0cf0-49b1-8e1b-3e641d73e724 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:57.873898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 819/1633] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:57 2022] PUT /identity/v3/projects/19d87d484a9f460e873612d2d77e54a8/users/07d1a207bc104128b123fa23a4493701/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:57.875069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd13c1d-519b-4cdd-9d32-f3c46bb21784 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:57.897098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 815/1634] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:59:57.899092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb6754-0cf0-49b1-8e1b-3e641d73e724 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:57.925249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb6754-0cf0-49b1-8e1b-3e641d73e724 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:57.963614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd13c1d-519b-4cdd-9d32-f3c46bb21784 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:57.969651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4615d47-46bd-4401-acf4-4ac26cb0fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:57.970533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4615d47-46bd-4401-acf4-4ac26cb0fc78 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.970922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4615d47-46bd-4401-acf4-4ac26cb0fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.971867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4615d47-46bd-4401-acf4-4ac26cb0fc78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:57.973569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4615d47-46bd-4401-acf4-4ac26cb0fc78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:57.975433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4615d47-46bd-4401-acf4-4ac26cb0fc78 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:57.993769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 820/1635] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:57.995923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f538a6-0851-4195-8638-5e9eabdfa1c0 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:57.996351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f538a6-0851-4195-8638-5e9eabdfa1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:57.996973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f538a6-0851-4195-8638-5e9eabdfa1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:58.017844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:58.051193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e2f538a6-0851-4195-8638-5e9eabdfa1c0 None None] MFA Rules not processed for user `07d1a207bc104128b123fa23a4493701`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:58.056272 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e2f538a6-0851-4195-8638-5e9eabdfa1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:58.077886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ed526-867f-46b5-91e3-2faf6fac770b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:58.079948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ed526-867f-46b5-91e3-2faf6fac770b service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:58.080245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ed526-867f-46b5-91e3-2faf6fac770b service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:58.080641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ed526-867f-46b5-91e3-2faf6fac770b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:58.142825 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:58.224204 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ed526-867f-46b5-91e3-2faf6fac770b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:58.226095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ed526-867f-46b5-91e3-2faf6fac770b service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:58.233737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 821/1636] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:58.234951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f538a6-0851-4195-8638-5e9eabdfa1c0 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:58.248542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 816/1637] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:59:58.264269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f538a6-0851-4195-8638-5e9eabdfa1c0 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:58.267923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ed526-867f-46b5-91e3-2faf6fac770b service nova] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:58.277152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f66809-75cf-4143-a476-798c91883933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:58.278575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f66809-75cf-4143-a476-798c91883933 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:58.278950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f66809-75cf-4143-a476-798c91883933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:58.281103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f66809-75cf-4143-a476-798c91883933 admin admin] PATH_INFO: `/v3/projects/a530fa9155dd449990dc061c5cde4747/users/bcf52210bb4a43f5846b5da2efe36876/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:58.318062 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f66809-75cf-4143-a476-798c91883933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a530fa9155dd449990dc061c5cde4747, user_id=bcf52210bb4a43f5846b5da2efe36876, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:58.321240 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f66809-75cf-4143-a476-798c91883933 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:58.350300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 822/1638] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:58 2022] PUT /identity/v3/projects/a530fa9155dd449990dc061c5cde4747/users/bcf52210bb4a43f5846b5da2efe36876/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:58.354693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f66809-75cf-4143-a476-798c91883933 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:58.467687 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ed526-867f-46b5-91e3-2faf6fac770b service nova] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:58.512675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f66809-75cf-4143-a476-798c91883933 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:58.535763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2511b-9bc2-447f-8e4f-304462f15825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:58.536477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2511b-9bc2-447f-8e4f-304462f15825 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:58.536771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2511b-9bc2-447f-8e4f-304462f15825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:58.537055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2511b-9bc2-447f-8e4f-304462f15825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:58.537807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2511b-9bc2-447f-8e4f-304462f15825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:58.539494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2511b-9bc2-447f-8e4f-304462f15825 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:58.577742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 823/1639] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 12:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 12:59:58.581219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2511b-9bc2-447f-8e4f-304462f15825 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:58.589237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5a79a-7e4f-4ec4-a187-24dab6fcf19c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:58.590181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5a79a-7e4f-4ec4-a187-24dab6fcf19c service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:58.590822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5a79a-7e4f-4ec4-a187-24dab6fcf19c service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:58.591261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5a79a-7e4f-4ec4-a187-24dab6fcf19c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:58.603945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2511b-9bc2-447f-8e4f-304462f15825 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:58.616637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1027e83-88f9-413f-8f1f-3a39ddec774e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:58.619978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1027e83-88f9-413f-8f1f-3a39ddec774e admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:58.620523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1027e83-88f9-413f-8f1f-3a39ddec774e admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:58.621999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1027e83-88f9-413f-8f1f-3a39ddec774e admin admin] PATH_INFO: `/v3/projects/a530fa9155dd449990dc061c5cde4747/users/bcf52210bb4a43f5846b5da2efe36876/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:58.628041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1027e83-88f9-413f-8f1f-3a39ddec774e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a530fa9155dd449990dc061c5cde4747, user_id=bcf52210bb4a43f5846b5da2efe36876, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:58.635094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1027e83-88f9-413f-8f1f-3a39ddec774e admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:58.661971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 824/1640] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 12:59:58 2022] PUT /identity/v3/projects/a530fa9155dd449990dc061c5cde4747/users/bcf52210bb4a43f5846b5da2efe36876/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 12:59:58.674238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d38e19-27ca-465e-9cf1-0a02c2b5cbc6 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:58.674567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d38e19-27ca-465e-9cf1-0a02c2b5cbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:58.674870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d38e19-27ca-465e-9cf1-0a02c2b5cbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:58.704514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:58.728678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e1d38e19-27ca-465e-9cf1-0a02c2b5cbc6 None None] MFA Rules not processed for user `bcf52210bb4a43f5846b5da2efe36876`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:58.730525 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e1d38e19-27ca-465e-9cf1-0a02c2b5cbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:58.734870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5a79a-7e4f-4ec4-a187-24dab6fcf19c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:58.736328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5a79a-7e4f-4ec4-a187-24dab6fcf19c service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:58.765032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 817/1641] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 12:59:58.832919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 825/1642] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 12:59:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 12:59:58.843053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074fb16-b0fb-4c34-a93c-56a3a247dc47 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:58.843491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074fb16-b0fb-4c34-a93c-56a3a247dc47 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:58.843966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074fb16-b0fb-4c34-a93c-56a3a247dc47 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:58.845678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 818/1643] 10.222.0.45 () {54 vars in 912 bytes} [Tue May 24 12:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:59:58.853000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4485cc-f744-41b8-951e-87a9fd677b4f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:58.853512 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4485cc-f744-41b8-951e-87a9fd677b4f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:58.854632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4485cc-f744-41b8-951e-87a9fd677b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:58.878890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8a4485cc-f744-41b8-951e-87a9fd677b4f None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:58.880569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8a4485cc-f744-41b8-951e-87a9fd677b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:58.895621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074fb16-b0fb-4c34-a93c-56a3a247dc47 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:58.903778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 12:59:59.013215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 826/1644] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 12:59:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:59.015746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074fb16-b0fb-4c34-a93c-56a3a247dc47 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:59.024219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4485cc-f744-41b8-951e-87a9fd677b4f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:59.025707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8a4485cc-f744-41b8-951e-87a9fd677b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:59.027616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2afc01-50d1-4e72-8772-092d37c514eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:59.028436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2afc01-50d1-4e72-8772-092d37c514eb service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:59.028734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2afc01-50d1-4e72-8772-092d37c514eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:59.029020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2afc01-50d1-4e72-8772-092d37c514eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:59.053235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2afc01-50d1-4e72-8772-092d37c514eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:59.054776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2afc01-50d1-4e72-8772-092d37c514eb service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:59.096219 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 819/1645] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 12:59:59.177686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4485cc-f744-41b8-951e-87a9fd677b4f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:59.189659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ffa22-90b1-46f0-91be-c8faecd0db05 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e47f1d65687e4cfcbce764076141702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:59.190716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ffa22-90b1-46f0-91be-c8faecd0db05 service glance] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:59.192548 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ffa22-90b1-46f0-91be-c8faecd0db05 service glance] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:59.192903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ffa22-90b1-46f0-91be-c8faecd0db05 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:59.217403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ffa22-90b1-46f0-91be-c8faecd0db05 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:59.219387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ffa22-90b1-46f0-91be-c8faecd0db05 service glance] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:59.241590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 827/1646] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 12:59:59.295259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2afc01-50d1-4e72-8772-092d37c514eb service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:59.310542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2afc01-50d1-4e72-8772-092d37c514eb service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:59.323457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8b382-7833-48be-8b84-05910eb221e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:59.324661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8b382-7833-48be-8b84-05910eb221e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:59.325088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8b382-7833-48be-8b84-05910eb221e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:59.325603 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8b382-7833-48be-8b84-05910eb221e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:59.444585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8b382-7833-48be-8b84-05910eb221e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:59.446059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8b382-7833-48be-8b84-05910eb221e3 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:59.458010 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 820/1647] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 12:59:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 12:59:59.732595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ffa22-90b1-46f0-91be-c8faecd0db05 service glance] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 12:59:59.843069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbac1b8-21cc-49c7-a12b-109c28674b73 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:59.843366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbac1b8-21cc-49c7-a12b-109c28674b73 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:59.843650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbac1b8-21cc-49c7-a12b-109c28674b73 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:59.845819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 821/1648] 10.222.0.45 () {54 vars in 912 bytes} [Tue May 24 12:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 12:59:59.850866 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695a16e-7b9b-49d9-9ca6-44b7b684a5aa None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:59.851090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695a16e-7b9b-49d9-9ca6-44b7b684a5aa None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:59.851327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695a16e-7b9b-49d9-9ca6-44b7b684a5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:59.855236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ffa22-90b1-46f0-91be-c8faecd0db05 service glance] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 12:59:59.867617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 12:59:59.870197 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 12:59:59.872350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 12:59:59.873398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] PATH_INFO: `/v3/users/65040649551741d5ad79ead288213a32` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 12:59:59.876887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65040649551741d5ad79ead288213a32)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 12:59:59.879722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: WARNING py.warnings [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 12:59:59.879722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: warnings.warn(msg) May 24 12:59:59.879722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: May 24 12:59:59.881054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-d695a16e-7b9b-49d9-9ca6-44b7b684a5aa None None] MFA Rules not processed for user `bb111be6bc2946d3b50b8d99a89cf8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 12:59:59.882331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d695a16e-7b9b-49d9-9ca6-44b7b684a5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 12:59:59.883980 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 12:59:59.989191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 822/1649] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 12:59:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 12:59:59.996332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65040649551741d5ad79ead288213a32'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:00.001085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695a16e-7b9b-49d9-9ca6-44b7b684a5aa None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:00.003058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:00.015102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65040649551741d5ad79ead288213a32'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:00.016253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65040649551741d5ad79ead288213a32'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:00.017485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65040649551741d5ad79ead288213a32'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:00.018335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695a16e-7b9b-49d9-9ca6-44b7b684a5aa None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:00.023913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab818c-c478-43fb-8ff0-9f9c69f64468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:00.025560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab818c-c478-43fb-8ff0-9f9c69f64468 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:00.026913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab818c-c478-43fb-8ff0-9f9c69f64468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:00.026913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab818c-c478-43fb-8ff0-9f9c69f64468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:00.029159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a1ab818c-c478-43fb-8ff0-9f9c69f64468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:00.047371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65040649551741d5ad79ead288213a32'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:00.049278 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab818c-c478-43fb-8ff0-9f9c69f64468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:00.053348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab818c-c478-43fb-8ff0-9f9c69f64468 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:00.071171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 823/1650] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:00.079762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 828/1651] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 12:59:59 2022] DELETE /identity/v3/users/65040649551741d5ad79ead288213a32 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:00.081384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab1af3-31cb-48ab-bfa7-0ae3872270f3 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:00.084121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab1af3-31cb-48ab-bfa7-0ae3872270f3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:00.084428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab1af3-31cb-48ab-bfa7-0ae3872270f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:00.121531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e8ab1af3-31cb-48ab-bfa7-0ae3872270f3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:00.123486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e8ab1af3-31cb-48ab-bfa7-0ae3872270f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:00.153623 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:00.236080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 824/1652] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:00.251371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:00.253825 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:00.386388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fccb4e-1ef1-4795-9b73-63398badcb14 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:00.391299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a671a4-e8d3-4743-8330-024f502f42f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:00.392815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a671a4-e8d3-4743-8330-024f502f42f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:00.393140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a671a4-e8d3-4743-8330-024f502f42f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:00.393529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a671a4-e8d3-4743-8330-024f502f42f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:00.416491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a671a4-e8d3-4743-8330-024f502f42f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:00.418914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a671a4-e8d3-4743-8330-024f502f42f8 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:00.434782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 829/1653] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:01.056761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840399db-e660-4ef6-ba3d-2eace3499009 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:01.057315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840399db-e660-4ef6-ba3d-2eace3499009 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:01.057588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840399db-e660-4ef6-ba3d-2eace3499009 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:01.061630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 825/1654] 10.222.0.45 () {54 vars in 960 bytes} [Tue May 24 13:00:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:00:01.066919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be330c08-2873-4807-9432-29d20a00c773 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:01.067445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be330c08-2873-4807-9432-29d20a00c773 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:01.067547 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be330c08-2873-4807-9432-29d20a00c773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:01.107114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-be330c08-2873-4807-9432-29d20a00c773 None None] MFA Rules not processed for user `630f0527bd844de9be072c459dab96c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:01.108391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-be330c08-2873-4807-9432-29d20a00c773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:01.270891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 830/1655] 10.222.0.45 () {58 vars in 1057 bytes} [Tue May 24 13:00:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:00:01.281003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840399db-e660-4ef6-ba3d-2eace3499009 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:01.301843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840399db-e660-4ef6-ba3d-2eace3499009 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:01.319686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae88a7-9ae9-495d-9144-ee3ebe28b9de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:01.320352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae88a7-9ae9-495d-9144-ee3ebe28b9de service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:01.320633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae88a7-9ae9-495d-9144-ee3ebe28b9de service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:01.321100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae88a7-9ae9-495d-9144-ee3ebe28b9de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:01.324836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-cbae88a7-9ae9-495d-9144-ee3ebe28b9de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:01.352323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae88a7-9ae9-495d-9144-ee3ebe28b9de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:01.354078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae88a7-9ae9-495d-9144-ee3ebe28b9de service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:01.395905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 826/1656] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:01.643559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be330c08-2873-4807-9432-29d20a00c773 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:01.645080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-be330c08-2873-4807-9432-29d20a00c773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:01.674861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be330c08-2873-4807-9432-29d20a00c773 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:01.707071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:01.713245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:01.713649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:01.714007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] PATH_INFO: `/v3/projects/ec3ca507ef134b528c0469eee600b0c6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:01.717477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec3ca507ef134b528c0469eee600b0c6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:01.727029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:01.865776 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec3ca507ef134b528c0469eee600b0c6', 'request_id': 'req-cf7612d8-c532-4da2-a9d7-b90a5df0121e'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:01.882729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec3ca507ef134b528c0469eee600b0c6', 'request_id': 'req-cf7612d8-c532-4da2-a9d7-b90a5df0121e'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:01.893807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:01.901748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec3ca507ef134b528c0469eee600b0c6', 'request_id': 'req-cf7612d8-c532-4da2-a9d7-b90a5df0121e'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:01.957110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] The token cache is being invalidate because project ec3ca507ef134b528c0469eee600b0c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:01.957110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:01.971812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:01.981320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae88a7-9ae9-495d-9144-ee3ebe28b9de service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:01.981320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-cbae88a7-9ae9-495d-9144-ee3ebe28b9de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:01.989519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 831/1657] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:01 2022] DELETE /identity/v3/projects/ec3ca507ef134b528c0469eee600b0c6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:01.990737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:01.997771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:02.008738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae88a7-9ae9-495d-9144-ee3ebe28b9de service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:02.182649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece67ab9-a70e-4642-9a40-a299e51494c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:02.184428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece67ab9-a70e-4642-9a40-a299e51494c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:02.185232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece67ab9-a70e-4642-9a40-a299e51494c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:02.185953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece67ab9-a70e-4642-9a40-a299e51494c0 admin admin] PATH_INFO: `/v3/users/ef208b9fd08940a180e3468ecd4e821f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:02.188997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece67ab9-a70e-4642-9a40-a299e51494c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef208b9fd08940a180e3468ecd4e821f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:02.192196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece67ab9-a70e-4642-9a40-a299e51494c0 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:02.219150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7612d8-c532-4da2-a9d7-b90a5df0121e admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:02.234955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:02.237198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:02.237807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:02.240301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] PATH_INFO: `/v3/users/07d1a207bc104128b123fa23a4493701` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:02.243184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07d1a207bc104128b123fa23a4493701)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:02.252078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:02.324049 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ece67ab9-a70e-4642-9a40-a299e51494c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef208b9fd08940a180e3468ecd4e821f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:02.394383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ece67ab9-a70e-4642-9a40-a299e51494c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef208b9fd08940a180e3468ecd4e821f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:02.412012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07d1a207bc104128b123fa23a4493701'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:02.421143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ece67ab9-a70e-4642-9a40-a299e51494c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef208b9fd08940a180e3468ecd4e821f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:02.422974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ece67ab9-a70e-4642-9a40-a299e51494c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef208b9fd08940a180e3468ecd4e821f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:02.428560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07d1a207bc104128b123fa23a4493701'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:02.429691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07d1a207bc104128b123fa23a4493701'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:02.430575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07d1a207bc104128b123fa23a4493701'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:02.445095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ece67ab9-a70e-4642-9a40-a299e51494c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef208b9fd08940a180e3468ecd4e821f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:02.458832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 827/1658] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:01 2022] DELETE /identity/v3/users/ef208b9fd08940a180e3468ecd4e821f => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:02.463655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289bd75-9529-47a9-af46-c5818f79d841 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:02.468646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289bd75-9529-47a9-af46-c5818f79d841 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:02.469077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289bd75-9529-47a9-af46-c5818f79d841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:02.502073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07d1a207bc104128b123fa23a4493701'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:02.517398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5289bd75-9529-47a9-af46-c5818f79d841 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:02.525235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5289bd75-9529-47a9-af46-c5818f79d841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:02.543028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 832/1659] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:01 2022] DELETE /identity/v3/users/07d1a207bc104128b123fa23a4493701 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:02.617338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:02.705282 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 828/1660] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:02.736855 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:02.738600 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:03.052680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4890219-e9af-43e5-a5a7-032271208b99 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:03.071972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df944c2a-7e87-40b7-a7a5-5d53552755e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:03.073210 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df944c2a-7e87-40b7-a7a5-5d53552755e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:03.073543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df944c2a-7e87-40b7-a7a5-5d53552755e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:03.073824 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df944c2a-7e87-40b7-a7a5-5d53552755e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:03.116112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:03.194576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df944c2a-7e87-40b7-a7a5-5d53552755e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:03.196135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df944c2a-7e87-40b7-a7a5-5d53552755e7 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:03.210269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 833/1661] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:04.766537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289bd75-9529-47a9-af46-c5818f79d841 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:04.770722 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5289bd75-9529-47a9-af46-c5818f79d841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:04.776666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df944c2a-7e87-40b7-a7a5-5d53552755e7 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:04.778096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-df944c2a-7e87-40b7-a7a5-5d53552755e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:04.796479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289bd75-9529-47a9-af46-c5818f79d841 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:04.809692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df944c2a-7e87-40b7-a7a5-5d53552755e7 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:04.839177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:04.840286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:04.840713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:04.841136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] PATH_INFO: `/v3/projects/19d87d484a9f460e873612d2d77e54a8` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:04.842421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:04.843078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d87d484a9f460e873612d2d77e54a8)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:04.844944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:04.846652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:04.848360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:04.849337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] PATH_INFO: `/v3/projects/30d674774c41482ab88fb61053f56c35` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:04.852863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30d674774c41482ab88fb61053f56c35)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:04.858463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: WARNING py.warnings [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:00:04.858463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: warnings.warn(msg) May 24 13:00:04.858463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: May 24 13:00:04.867774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:05.105148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30d674774c41482ab88fb61053f56c35', 'request_id': 'req-898efb31-4c7f-49a6-b658-0694a9fef8f7'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:05.132573 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30d674774c41482ab88fb61053f56c35', 'request_id': 'req-898efb31-4c7f-49a6-b658-0694a9fef8f7'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:05.152372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30d674774c41482ab88fb61053f56c35', 'request_id': 'req-898efb31-4c7f-49a6-b658-0694a9fef8f7'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:05.224653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] The token cache is being invalidate because project 30d674774c41482ab88fb61053f56c35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:05.225137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:05.234733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-898efb31-4c7f-49a6-b658-0694a9fef8f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:05.237277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 829/1662] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:04 2022] DELETE /identity/v3/projects/30d674774c41482ab88fb61053f56c35 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:05.244275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac757-8ab4-46e9-9406-d11ee614e91e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:05.244631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac757-8ab4-46e9-9406-d11ee614e91e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:05.244913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac757-8ab4-46e9-9406-d11ee614e91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:05.318278 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-f55ac757-8ab4-46e9-9406-d11ee614e91e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:05.322957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f55ac757-8ab4-46e9-9406-d11ee614e91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:05.438730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d87d484a9f460e873612d2d77e54a8', 'request_id': 'req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:05.453713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d87d484a9f460e873612d2d77e54a8', 'request_id': 'req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:05.460833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d87d484a9f460e873612d2d77e54a8', 'request_id': 'req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:05.475029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] The token cache is being invalidate because project 19d87d484a9f460e873612d2d77e54a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:05.475328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:05.475999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9691e7a4-ce72-4a3e-a95d-a8a7a307adb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:05.477403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 830/1663] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:05.482509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 834/1664] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:04 2022] DELETE /identity/v3/projects/19d87d484a9f460e873612d2d77e54a8 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:05.483758 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4e97e-ee39-41fc-bcde-5ad4224ab255 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:05.484058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4e97e-ee39-41fc-bcde-5ad4224ab255 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:05.484341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4e97e-ee39-41fc-bcde-5ad4224ab255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:05.486651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e711a-d70c-4b39-9f56-9bb8a2fa8d7e None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:05.487679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e711a-d70c-4b39-9f56-9bb8a2fa8d7e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:05.488160 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e711a-d70c-4b39-9f56-9bb8a2fa8d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:05.530884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-1c9e711a-d70c-4b39-9f56-9bb8a2fa8d7e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:05.532893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-21e4e97e-ee39-41fc-bcde-5ad4224ab255 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:05.533423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1c9e711a-d70c-4b39-9f56-9bb8a2fa8d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:05.535936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-21e4e97e-ee39-41fc-bcde-5ad4224ab255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:05.650543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:05.658411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 835/1665] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:05.660160 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e711a-d70c-4b39-9f56-9bb8a2fa8d7e None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:05.683146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 831/1666] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:05.691317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10416d0a-3f7c-42ac-8878-faead7f44b2b None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:05.691701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10416d0a-3f7c-42ac-8878-faead7f44b2b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:05.691904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10416d0a-3f7c-42ac-8878-faead7f44b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:05.694272 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e711a-d70c-4b39-9f56-9bb8a2fa8d7e None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:05.702372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655114ec-f3d1-4ea1-aa40-62077e00f584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:05.704875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655114ec-f3d1-4ea1-aa40-62077e00f584 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:05.705818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655114ec-f3d1-4ea1-aa40-62077e00f584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:05.706178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655114ec-f3d1-4ea1-aa40-62077e00f584 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:05.708418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655114ec-f3d1-4ea1-aa40-62077e00f584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:00:05.708814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655114ec-f3d1-4ea1-aa40-62077e00f584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:05.716014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655114ec-f3d1-4ea1-aa40-62077e00f584 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:05.772453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 836/1667] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:00:05.774000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655114ec-f3d1-4ea1-aa40-62077e00f584 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:05.774881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-10416d0a-3f7c-42ac-8878-faead7f44b2b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:05.794209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-10416d0a-3f7c-42ac-8878-faead7f44b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:05.814673 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655114ec-f3d1-4ea1-aa40-62077e00f584 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:05.821719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 832/1668] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:05.822860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10416d0a-3f7c-42ac-8878-faead7f44b2b None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:05.836591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eba91e-e11d-4bf1-b354-de9f8ee19df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:05.839412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eba91e-e11d-4bf1-b354-de9f8ee19df8 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:05.839870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eba91e-e11d-4bf1-b354-de9f8ee19df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:05.840286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eba91e-e11d-4bf1-b354-de9f8ee19df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:05.843723 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eba91e-e11d-4bf1-b354-de9f8ee19df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:05.852781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eba91e-e11d-4bf1-b354-de9f8ee19df8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:05.855987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10416d0a-3f7c-42ac-8878-faead7f44b2b None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:05.882982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a87728-a8dd-43ac-90e1-74a46e445798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:05.884507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a87728-a8dd-43ac-90e1-74a46e445798 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:05.884937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a87728-a8dd-43ac-90e1-74a46e445798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:05.885535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a87728-a8dd-43ac-90e1-74a46e445798 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:05.886459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a87728-a8dd-43ac-90e1-74a46e445798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:00:05.886911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a87728-a8dd-43ac-90e1-74a46e445798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:05.892213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a87728-a8dd-43ac-90e1-74a46e445798 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:05.931246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 833/1669] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:00:05.932689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 837/1670] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:05 2022] POST /identity/v3/projects => generated 403 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:05.935565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eba91e-e11d-4bf1-b354-de9f8ee19df8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:05.936033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a87728-a8dd-43ac-90e1-74a46e445798 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:05.956796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:05.962561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a87728-a8dd-43ac-90e1-74a46e445798 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:06.141609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eba91e-e11d-4bf1-b354-de9f8ee19df8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:06.151406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9981f5-5921-40ae-ab71-17f37499f482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:06.152099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9981f5-5921-40ae-ab71-17f37499f482 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:06.152384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9981f5-5921-40ae-ab71-17f37499f482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:06.153656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9981f5-5921-40ae-ab71-17f37499f482 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:06.154574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc251fa3-c07c-4fb6-846d-b4713e000fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:06.155260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc251fa3-c07c-4fb6-846d-b4713e000fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:06.155544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc251fa3-c07c-4fb6-846d-b4713e000fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:06.155818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc251fa3-c07c-4fb6-846d-b4713e000fb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:06.157233 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9981f5-5921-40ae-ab71-17f37499f482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:06.158368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc251fa3-c07c-4fb6-846d-b4713e000fb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:06.163595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9981f5-5921-40ae-ab71-17f37499f482 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:06.165191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc251fa3-c07c-4fb6-846d-b4713e000fb8 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:06.231564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 834/1671] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:05 2022] POST /identity/v3/users => generated 302 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:06.234435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc251fa3-c07c-4fb6-846d-b4713e000fb8 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:06.252282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 838/1672] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:05 2022] POST /identity/v3/projects => generated 369 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:06.253840 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9981f5-5921-40ae-ab71-17f37499f482 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:06.265401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc251fa3-c07c-4fb6-846d-b4713e000fb8 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:06.399399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9981f5-5921-40ae-ab71-17f37499f482 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:06.416392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49880b-5a0d-46b9-afe7-3694ae583657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:06.417795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49880b-5a0d-46b9-afe7-3694ae583657 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:06.418087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49880b-5a0d-46b9-afe7-3694ae583657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:06.418391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49880b-5a0d-46b9-afe7-3694ae583657 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:06.419105 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49880b-5a0d-46b9-afe7-3694ae583657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:06.421443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49880b-5a0d-46b9-afe7-3694ae583657 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:06.446700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770543f-f514-4b3f-a9be-f2d4fc24217d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:06.452927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770543f-f514-4b3f-a9be-f2d4fc24217d admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:06.452927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770543f-f514-4b3f-a9be-f2d4fc24217d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:06.453139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770543f-f514-4b3f-a9be-f2d4fc24217d admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:06.456082 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770543f-f514-4b3f-a9be-f2d4fc24217d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:06.461825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770543f-f514-4b3f-a9be-f2d4fc24217d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:06.481730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 839/1673] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:06 2022] POST /identity/v3/users => generated 285 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:06.485962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49880b-5a0d-46b9-afe7-3694ae583657 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:06.507948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 835/1674] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:06.511239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770543f-f514-4b3f-a9be-f2d4fc24217d admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:06.519799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49880b-5a0d-46b9-afe7-3694ae583657 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:06.543194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770543f-f514-4b3f-a9be-f2d4fc24217d admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:06.544597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d62a8-acd4-4e7b-b58a-7512a9ef1a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:06.545464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d62a8-acd4-4e7b-b58a-7512a9ef1a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:06.545726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d62a8-acd4-4e7b-b58a-7512a9ef1a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:06.546012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d62a8-acd4-4e7b-b58a-7512a9ef1a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:06.546755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d62a8-acd4-4e7b-b58a-7512a9ef1a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:06.548393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d62a8-acd4-4e7b-b58a-7512a9ef1a16 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:06.555848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70d6d8-a38e-4f77-9797-5af36992f145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:06.556314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70d6d8-a38e-4f77-9797-5af36992f145 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:06.556605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70d6d8-a38e-4f77-9797-5af36992f145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:06.557413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70d6d8-a38e-4f77-9797-5af36992f145 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:06.558365 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70d6d8-a38e-4f77-9797-5af36992f145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:06.559784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70d6d8-a38e-4f77-9797-5af36992f145 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:06.570778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 836/1675] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:06.576093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70d6d8-a38e-4f77-9797-5af36992f145 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:06.583314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 840/1676] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:06.584415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d62a8-acd4-4e7b-b58a-7512a9ef1a16 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:06.610336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d62a8-acd4-4e7b-b58a-7512a9ef1a16 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:06.625022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b7135-c790-4877-9d9b-b7bde5bad6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:06.625721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7135-c790-4877-9d9b-b7bde5bad6ad admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:06.626007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7135-c790-4877-9d9b-b7bde5bad6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:06.626276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7135-c790-4877-9d9b-b7bde5bad6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:06.627026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b7135-c790-4877-9d9b-b7bde5bad6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:06.628638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b7135-c790-4877-9d9b-b7bde5bad6ad admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:06.631620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70d6d8-a38e-4f77-9797-5af36992f145 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:06.652622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 841/1677] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:00:06.654227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c6440-8dc4-439a-9ea6-e142ea9ea961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:06.655234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6440-8dc4-439a-9ea6-e142ea9ea961 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:06.655593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6440-8dc4-439a-9ea6-e142ea9ea961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:06.655881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6440-8dc4-439a-9ea6-e142ea9ea961 admin admin] PATH_INFO: `/v3/projects/f1e5a0af9bbb4755aa58218e2fb1fd66/users/7ac98870c6524e879027d31300a07f28/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:06.657734 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b7135-c790-4877-9d9b-b7bde5bad6ad admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:06.685370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b7135-c790-4877-9d9b-b7bde5bad6ad admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:06.702199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe38693-605d-42ee-ab6e-fccf5c1a011a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:06.704260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe38693-605d-42ee-ab6e-fccf5c1a011a admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:06.705480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe38693-605d-42ee-ab6e-fccf5c1a011a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:06.710161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c6440-8dc4-439a-9ea6-e142ea9ea961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e5a0af9bbb4755aa58218e2fb1fd66, user_id=7ac98870c6524e879027d31300a07f28, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:06.713347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c6440-8dc4-439a-9ea6-e142ea9ea961 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:06.716428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe38693-605d-42ee-ab6e-fccf5c1a011a admin admin] PATH_INFO: `/v3/projects/25dcf2e0f9c847eea87c8ee3df536f00/users/d499742744504dd48a53df233803834b/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:06.741170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe38693-605d-42ee-ab6e-fccf5c1a011a admin admin] RBAC: Authorizing `identity:create_grant(project_id=25dcf2e0f9c847eea87c8ee3df536f00, user_id=d499742744504dd48a53df233803834b, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:06.744949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 837/1678] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:06 2022] PUT /identity/v3/projects/f1e5a0af9bbb4755aa58218e2fb1fd66/users/7ac98870c6524e879027d31300a07f28/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:06.746932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe38693-605d-42ee-ab6e-fccf5c1a011a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:06.751672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5204e88-c52d-407d-8e9d-c663d4fb85e5 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:06.752024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5204e88-c52d-407d-8e9d-c663d4fb85e5 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:06.752239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5204e88-c52d-407d-8e9d-c663d4fb85e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:06.788335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 842/1679] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:06 2022] PUT /identity/v3/projects/25dcf2e0f9c847eea87c8ee3df536f00/users/d499742744504dd48a53df233803834b/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:06.793107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a5204e88-c52d-407d-8e9d-c663d4fb85e5 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:06.803366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02026b2-1adf-47f7-aa9a-ee6375c5c2f9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:06.803773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02026b2-1adf-47f7-aa9a-ee6375c5c2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:06.804251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02026b2-1adf-47f7-aa9a-ee6375c5c2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:06.811781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a5204e88-c52d-407d-8e9d-c663d4fb85e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:06.866759 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:06.879262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a02026b2-1adf-47f7-aa9a-ee6375c5c2f9 None None] MFA Rules not processed for user `d499742744504dd48a53df233803834b`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:06.894810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a02026b2-1adf-47f7-aa9a-ee6375c5c2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:07.019418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:07.052004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 838/1680] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:00:07.129039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 843/1681] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:07.133757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5204e88-c52d-407d-8e9d-c663d4fb85e5 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:07.146415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5204e88-c52d-407d-8e9d-c663d4fb85e5 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:07.152966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581014dc-c8a6-4ee3-9203-21334546a8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:07.153660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581014dc-c8a6-4ee3-9203-21334546a8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:07.153947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581014dc-c8a6-4ee3-9203-21334546a8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:07.154229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581014dc-c8a6-4ee3-9203-21334546a8d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:07.155161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581014dc-c8a6-4ee3-9203-21334546a8d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:07.157184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581014dc-c8a6-4ee3-9203-21334546a8d7 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:07.184878 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02026b2-1adf-47f7-aa9a-ee6375c5c2f9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:07.186738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a02026b2-1adf-47f7-aa9a-ee6375c5c2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:07.247895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 839/1682] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:07 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:00:07.252631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581014dc-c8a6-4ee3-9203-21334546a8d7 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:07.394556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02026b2-1adf-47f7-aa9a-ee6375c5c2f9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:07.461480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581014dc-c8a6-4ee3-9203-21334546a8d7 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:07.469137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251275cb-a1c0-4e43-8aef-e88a1d1061ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:07.470607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251275cb-a1c0-4e43-8aef-e88a1d1061ff admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:07.471374 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251275cb-a1c0-4e43-8aef-e88a1d1061ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:07.472328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251275cb-a1c0-4e43-8aef-e88a1d1061ff admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:07.473352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251275cb-a1c0-4e43-8aef-e88a1d1061ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:07.475881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251275cb-a1c0-4e43-8aef-e88a1d1061ff admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:07.520913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 840/1683] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:07 2022] POST /identity/v3/users => generated 290 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:07.527194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251275cb-a1c0-4e43-8aef-e88a1d1061ff admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:07.536489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657991ba-ab06-463c-ba23-fc55ddee3573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:07.537108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657991ba-ab06-463c-ba23-fc55ddee3573 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:07.537398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657991ba-ab06-463c-ba23-fc55ddee3573 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:07.537697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657991ba-ab06-463c-ba23-fc55ddee3573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:07.547989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251275cb-a1c0-4e43-8aef-e88a1d1061ff admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:07.557275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca71b6-75e1-4bc7-964b-d7c6e0f2f572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:07.557465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657991ba-ab06-463c-ba23-fc55ddee3573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:07.558432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca71b6-75e1-4bc7-964b-d7c6e0f2f572 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:07.558852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657991ba-ab06-463c-ba23-fc55ddee3573 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:07.559246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca71b6-75e1-4bc7-964b-d7c6e0f2f572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:07.559940 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca71b6-75e1-4bc7-964b-d7c6e0f2f572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:07.560894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca71b6-75e1-4bc7-964b-d7c6e0f2f572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:07.563416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca71b6-75e1-4bc7-964b-d7c6e0f2f572 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:07.580565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 844/1684] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:07.588341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 841/1685] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:07.592323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657991ba-ab06-463c-ba23-fc55ddee3573 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:07.606060 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657991ba-ab06-463c-ba23-fc55ddee3573 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:07.612381 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032e793-5766-4efb-93e3-0fbee94ffdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:07.613301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032e793-5766-4efb-93e3-0fbee94ffdf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:07.613582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032e793-5766-4efb-93e3-0fbee94ffdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:07.613853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032e793-5766-4efb-93e3-0fbee94ffdf4 admin admin] PATH_INFO: `/v3/projects/d62a2c4bb19842ecb1bbfd1a40c96f4d/users/1a537fcee20a4989806aef3864fed6bb/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:07.650217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032e793-5766-4efb-93e3-0fbee94ffdf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62a2c4bb19842ecb1bbfd1a40c96f4d, user_id=1a537fcee20a4989806aef3864fed6bb, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:07.653364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032e793-5766-4efb-93e3-0fbee94ffdf4 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:07.679579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 845/1686] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:07 2022] PUT /identity/v3/projects/d62a2c4bb19842ecb1bbfd1a40c96f4d/users/1a537fcee20a4989806aef3864fed6bb/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:07.683354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca71b6-75e1-4bc7-964b-d7c6e0f2f572 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:07.696056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca71b6-75e1-4bc7-964b-d7c6e0f2f572 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:07.836994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c0409-c064-4baf-b987-218e61b4a262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:07.837710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c0409-c064-4baf-b987-218e61b4a262 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:07.838026 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c0409-c064-4baf-b987-218e61b4a262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:07.838860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c0409-c064-4baf-b987-218e61b4a262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:07.846605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c0409-c064-4baf-b987-218e61b4a262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:07.848289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c0409-c064-4baf-b987-218e61b4a262 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:07.875799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032e793-5766-4efb-93e3-0fbee94ffdf4 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:07.882693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 842/1687] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:07.896821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032e793-5766-4efb-93e3-0fbee94ffdf4 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:07.912092 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac1b46-10ff-45d3-ad23-30d3f36178cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:07.914116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac1b46-10ff-45d3-ad23-30d3f36178cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:07.914116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac1b46-10ff-45d3-ad23-30d3f36178cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:07.914116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac1b46-10ff-45d3-ad23-30d3f36178cd admin admin] PATH_INFO: `/v3/projects/d62a2c4bb19842ecb1bbfd1a40c96f4d/users/1a537fcee20a4989806aef3864fed6bb/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:07.920566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac1b46-10ff-45d3-ad23-30d3f36178cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62a2c4bb19842ecb1bbfd1a40c96f4d, user_id=1a537fcee20a4989806aef3864fed6bb, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:07.923526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac1b46-10ff-45d3-ad23-30d3f36178cd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:07.961391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24425102-9b21-4113-8416-b35225bfbf38 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:07.961610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24425102-9b21-4113-8416-b35225bfbf38 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:07.961846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24425102-9b21-4113-8416-b35225bfbf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:07.963421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 846/1688] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:07 2022] PUT /identity/v3/projects/d62a2c4bb19842ecb1bbfd1a40c96f4d/users/1a537fcee20a4989806aef3864fed6bb/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:08.060610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-24425102-9b21-4113-8416-b35225bfbf38 None None] MFA Rules not processed for user `1a537fcee20a4989806aef3864fed6bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:08.062018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-24425102-9b21-4113-8416-b35225bfbf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:08.081869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:08.198436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 843/1689] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:08.206876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac1b46-10ff-45d3-ad23-30d3f36178cd admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:08.208921 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8aac1b46-10ff-45d3-ad23-30d3f36178cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:08.305600 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:08.319559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac1b46-10ff-45d3-ad23-30d3f36178cd admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:08.329327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3a23-9a14-4ba9-84bb-e563b73a8380 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:08.329931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3a23-9a14-4ba9-84bb-e563b73a8380 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:08.330214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3a23-9a14-4ba9-84bb-e563b73a8380 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:08.330848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3a23-9a14-4ba9-84bb-e563b73a8380 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:08.357596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b3a23-9a14-4ba9-84bb-e563b73a8380 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:08.359113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b3a23-9a14-4ba9-84bb-e563b73a8380 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:08.371616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 847/1690] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:08 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:09.332653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61fc9e-f12d-4b05-82a3-1215d66acab8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:09.333427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61fc9e-f12d-4b05-82a3-1215d66acab8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:09.333708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61fc9e-f12d-4b05-82a3-1215d66acab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:09.401914 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5f61fc9e-f12d-4b05-82a3-1215d66acab8 None None] MFA Rules not processed for user `f7b0dfb3d6514817aba4ab34fd972ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:09.408051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5f61fc9e-f12d-4b05-82a3-1215d66acab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:09.598730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 844/1691] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:09.605739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3a23-9a14-4ba9-84bb-e563b73a8380 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:09.608010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f38b3a23-9a14-4ba9-84bb-e563b73a8380 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:09.928384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3a23-9a14-4ba9-84bb-e563b73a8380 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:09.946145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71c43f-55e4-4c92-8a8d-51fd4ba91acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:09.948462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71c43f-55e4-4c92-8a8d-51fd4ba91acd service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:09.954954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71c43f-55e4-4c92-8a8d-51fd4ba91acd service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:09.955373 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71c43f-55e4-4c92-8a8d-51fd4ba91acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:09.989599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e434c8-2a0f-4d54-af9a-eaaf813f3372 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:09.990174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e434c8-2a0f-4d54-af9a-eaaf813f3372 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:09.990842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e434c8-2a0f-4d54-af9a-eaaf813f3372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:09.998598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71c43f-55e4-4c92-8a8d-51fd4ba91acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:10.003013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71c43f-55e4-4c92-8a8d-51fd4ba91acd service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:10.041902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-60e434c8-2a0f-4d54-af9a-eaaf813f3372 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:10.048542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 848/1692] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:00:10.050276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-60e434c8-2a0f-4d54-af9a-eaaf813f3372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:10.231303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 845/1693] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 13:00:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:10.522583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71c43f-55e4-4c92-8a8d-51fd4ba91acd service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:10.527718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4f71c43f-55e4-4c92-8a8d-51fd4ba91acd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:10.795768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71c43f-55e4-4c92-8a8d-51fd4ba91acd service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:10.812625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9101683-3cb7-4b60-b535-dc4a87fe510b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:10.813712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9101683-3cb7-4b60-b535-dc4a87fe510b service swift] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:10.814940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9101683-3cb7-4b60-b535-dc4a87fe510b service swift] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:10.815455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9101683-3cb7-4b60-b535-dc4a87fe510b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:10.838756 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9101683-3cb7-4b60-b535-dc4a87fe510b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:10.841815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9101683-3cb7-4b60-b535-dc4a87fe510b service swift] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:10.860233 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 849/1694] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:00:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:00:11.699325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e434c8-2a0f-4d54-af9a-eaaf813f3372 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:11.700856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-60e434c8-2a0f-4d54-af9a-eaaf813f3372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:11.917722 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:12.006152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e434c8-2a0f-4d54-af9a-eaaf813f3372 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:12.022939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476cfa0-f6ae-4f8c-8d57-e15f66e77d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:12.024029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476cfa0-f6ae-4f8c-8d57-e15f66e77d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:12.026763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476cfa0-f6ae-4f8c-8d57-e15f66e77d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:12.027270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476cfa0-f6ae-4f8c-8d57-e15f66e77d72 admin admin] PATH_INFO: `/v3/users/de8e37ee2be84839b16bdfc1f55004f9` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:12.030336 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476cfa0-f6ae-4f8c-8d57-e15f66e77d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de8e37ee2be84839b16bdfc1f55004f9)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:12.038695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476cfa0-f6ae-4f8c-8d57-e15f66e77d72 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:12.213509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9101683-3cb7-4b60-b535-dc4a87fe510b service swift] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:12.228366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9101683-3cb7-4b60-b535-dc4a87fe510b service swift] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:12.230239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4476cfa0-f6ae-4f8c-8d57-e15f66e77d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de8e37ee2be84839b16bdfc1f55004f9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:12.239468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f92a45-11d9-4b08-8e54-71bb0c94bd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:12.240642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f92a45-11d9-4b08-8e54-71bb0c94bd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:12.240932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f92a45-11d9-4b08-8e54-71bb0c94bd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:12.241208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f92a45-11d9-4b08-8e54-71bb0c94bd72 admin admin] PATH_INFO: `/v3/users/d499742744504dd48a53df233803834b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:12.244875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f92a45-11d9-4b08-8e54-71bb0c94bd72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d499742744504dd48a53df233803834b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:12.249362 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f92a45-11d9-4b08-8e54-71bb0c94bd72 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:12.307785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4476cfa0-f6ae-4f8c-8d57-e15f66e77d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de8e37ee2be84839b16bdfc1f55004f9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:12.325395 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4476cfa0-f6ae-4f8c-8d57-e15f66e77d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de8e37ee2be84839b16bdfc1f55004f9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:12.326043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4476cfa0-f6ae-4f8c-8d57-e15f66e77d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de8e37ee2be84839b16bdfc1f55004f9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:12.347038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4476cfa0-f6ae-4f8c-8d57-e15f66e77d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de8e37ee2be84839b16bdfc1f55004f9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:12.363884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 846/1695] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:11 2022] DELETE /identity/v3/users/de8e37ee2be84839b16bdfc1f55004f9 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:12.368830 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98bd88-2510-4582-bf6c-30c6fc55603b None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:12.369189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98bd88-2510-4582-bf6c-30c6fc55603b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:12.369472 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98bd88-2510-4582-bf6c-30c6fc55603b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:12.419207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-90f92a45-11d9-4b08-8e54-71bb0c94bd72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd499742744504dd48a53df233803834b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:12.433489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-90f92a45-11d9-4b08-8e54-71bb0c94bd72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd499742744504dd48a53df233803834b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:12.434222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-90f92a45-11d9-4b08-8e54-71bb0c94bd72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd499742744504dd48a53df233803834b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:12.435070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-90f92a45-11d9-4b08-8e54-71bb0c94bd72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd499742744504dd48a53df233803834b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:12.442778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-ea98bd88-2510-4582-bf6c-30c6fc55603b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:12.447315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ea98bd88-2510-4582-bf6c-30c6fc55603b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:12.453562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:12.478513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-90f92a45-11d9-4b08-8e54-71bb0c94bd72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd499742744504dd48a53df233803834b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:12.516989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 850/1696] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:12 2022] DELETE /identity/v3/users/d499742744504dd48a53df233803834b => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:12.522422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674620-fa61-4e2d-a632-1fc42bfa35dd None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:12.522796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674620-fa61-4e2d-a632-1fc42bfa35dd None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:12.523144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674620-fa61-4e2d-a632-1fc42bfa35dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:12.578526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-40674620-fa61-4e2d-a632-1fc42bfa35dd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:12.580763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-40674620-fa61-4e2d-a632-1fc42bfa35dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:12.663058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 847/1697] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:12.691191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98bd88-2510-4582-bf6c-30c6fc55603b None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:12.692925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ea98bd88-2510-4582-bf6c-30c6fc55603b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:12.770818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 851/1698] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:00:12.821410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40674620-fa61-4e2d-a632-1fc42bfa35dd None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:12.823954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-40674620-fa61-4e2d-a632-1fc42bfa35dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:12.831823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98bd88-2510-4582-bf6c-30c6fc55603b None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:12.851862 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:12.951406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5244c-45be-49ea-8b20-b756307343a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:12.952310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5244c-45be-49ea-8b20-b756307343a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:12.952753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5244c-45be-49ea-8b20-b756307343a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:12.953173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5244c-45be-49ea-8b20-b756307343a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:12.975200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5244c-45be-49ea-8b20-b756307343a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:12.977075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5244c-45be-49ea-8b20-b756307343a6 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:12.989727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 848/1699] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:00:13.040335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40674620-fa61-4e2d-a632-1fc42bfa35dd None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:13.069920 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6096efe-1038-4405-8eab-6f7e912a1553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:13.075704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6096efe-1038-4405-8eab-6f7e912a1553 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:13.076035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6096efe-1038-4405-8eab-6f7e912a1553 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:13.076324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6096efe-1038-4405-8eab-6f7e912a1553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:13.109813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6096efe-1038-4405-8eab-6f7e912a1553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:13.115946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6096efe-1038-4405-8eab-6f7e912a1553 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:13.164750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 852/1700] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:14.781516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5244c-45be-49ea-8b20-b756307343a6 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:14.783584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-bfe5244c-45be-49ea-8b20-b756307343a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:14.810881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5244c-45be-49ea-8b20-b756307343a6 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:14.826101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:14.830781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:14.831725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:14.832164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] PATH_INFO: `/v3/projects/25dcf2e0f9c847eea87c8ee3df536f00` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:14.836999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=25dcf2e0f9c847eea87c8ee3df536f00)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:14.842502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:14.995651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25dcf2e0f9c847eea87c8ee3df536f00', 'request_id': 'req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.011916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25dcf2e0f9c847eea87c8ee3df536f00', 'request_id': 'req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.037520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25dcf2e0f9c847eea87c8ee3df536f00', 'request_id': 'req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.079120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57b443-86f6-473e-b2e7-fd4ed3a53e98 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:15.079725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57b443-86f6-473e-b2e7-fd4ed3a53e98 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:15.079966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57b443-86f6-473e-b2e7-fd4ed3a53e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:15.110231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] The token cache is being invalidate because project 25dcf2e0f9c847eea87c8ee3df536f00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:15.110562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.113495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.123537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 849/1701] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:14 2022] DELETE /identity/v3/projects/25dcf2e0f9c847eea87c8ee3df536f00 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:15.124559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:15.142223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:15.208159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-fb57b443-86f6-473e-b2e7-fd4ed3a53e98 None None] MFA Rules not processed for user `cb3d46a7a0ae40e9b157b7199b18de27`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:15.212181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-fb57b443-86f6-473e-b2e7-fd4ed3a53e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:15.413649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc33b08-dbc0-4dd2-a8c0-916fc9ad87ab admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:15.430805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:15.433835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:15.434380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:15.435721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] PATH_INFO: `/v3/projects/c9a85b3254d14f45a87f6d58df821d4b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:15.454738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9a85b3254d14f45a87f6d58df821d4b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:15.458830 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:15.534982 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 853/1702] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:15.537150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57b443-86f6-473e-b2e7-fd4ed3a53e98 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:15.540017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-fb57b443-86f6-473e-b2e7-fd4ed3a53e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:15.589318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57b443-86f6-473e-b2e7-fd4ed3a53e98 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:15.617523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df296424-bc0f-42b0-8eee-ac14113fa82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:15.623552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296424-bc0f-42b0-8eee-ac14113fa82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:15.624062 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296424-bc0f-42b0-8eee-ac14113fa82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:15.624445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296424-bc0f-42b0-8eee-ac14113fa82f admin admin] PATH_INFO: `/v3/users/1a537fcee20a4989806aef3864fed6bb` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:15.629501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df296424-bc0f-42b0-8eee-ac14113fa82f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a537fcee20a4989806aef3864fed6bb)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:15.633430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df296424-bc0f-42b0-8eee-ac14113fa82f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:15.635439 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:15.648655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9a85b3254d14f45a87f6d58df821d4b', 'request_id': 'req-c7630e7b-2e6c-4d23-850c-cf5056081317'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.651329 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:15.665793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9a85b3254d14f45a87f6d58df821d4b', 'request_id': 'req-c7630e7b-2e6c-4d23-850c-cf5056081317'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.690273 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9a85b3254d14f45a87f6d58df821d4b', 'request_id': 'req-c7630e7b-2e6c-4d23-850c-cf5056081317'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.715482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] The token cache is being invalidate because project c9a85b3254d14f45a87f6d58df821d4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:15.715482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.716090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.718263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 850/1703] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:15 2022] DELETE /identity/v3/projects/c9a85b3254d14f45a87f6d58df821d4b => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:15.719055 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:15.720415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:15.796872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-df296424-bc0f-42b0-8eee-ac14113fa82f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a537fcee20a4989806aef3864fed6bb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.820670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-df296424-bc0f-42b0-8eee-ac14113fa82f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a537fcee20a4989806aef3864fed6bb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.822995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-df296424-bc0f-42b0-8eee-ac14113fa82f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a537fcee20a4989806aef3864fed6bb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.827758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-df296424-bc0f-42b0-8eee-ac14113fa82f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a537fcee20a4989806aef3864fed6bb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.880158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-df296424-bc0f-42b0-8eee-ac14113fa82f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a537fcee20a4989806aef3864fed6bb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:15.904407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 854/1704] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:15 2022] DELETE /identity/v3/users/1a537fcee20a4989806aef3864fed6bb => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:15.906088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcd32c-625e-45c9-a998-cd81bdd62c7a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:15.906624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcd32c-625e-45c9-a998-cd81bdd62c7a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:15.907150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcd32c-625e-45c9-a998-cd81bdd62c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:15.977770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-97fcd32c-625e-45c9-a998-cd81bdd62c7a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:15.982983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-97fcd32c-625e-45c9-a998-cd81bdd62c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:16.007824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7630e7b-2e6c-4d23-850c-cf5056081317 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:16.216369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 855/1705] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:16.224701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbb42e-ca95-4335-87c4-1a2939924512 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:16.225114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbb42e-ca95-4335-87c4-1a2939924512 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:16.225685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbb42e-ca95-4335-87c4-1a2939924512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:16.226238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b95aae-67ff-4bc3-bbe5-21dad48dd6ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:16.226943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b95aae-67ff-4bc3-bbe5-21dad48dd6ee service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:16.227228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b95aae-67ff-4bc3-bbe5-21dad48dd6ee service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:16.227651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b95aae-67ff-4bc3-bbe5-21dad48dd6ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:16.233729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-09b95aae-67ff-4bc3-bbe5-21dad48dd6ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:16.287895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e8dbb42e-ca95-4335-87c4-1a2939924512 None None] MFA Rules not processed for user `bb111be6bc2946d3b50b8d99a89cf8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:16.287895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e8dbb42e-ca95-4335-87c4-1a2939924512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:16.404838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b95aae-67ff-4bc3-bbe5-21dad48dd6ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:16.408016 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b95aae-67ff-4bc3-bbe5-21dad48dd6ee service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:16.424500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 856/1706] 10.222.0.13 () {58 vars in 1015 bytes} [Tue May 24 13:00:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:16.425894 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3738ae3-8f92-4dd4-8fbc-9a4dafa8f5f8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:16.426520 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3738ae3-8f92-4dd4-8fbc-9a4dafa8f5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:16.426896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3738ae3-8f92-4dd4-8fbc-9a4dafa8f5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:16.433889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 851/1707] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:16.439000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e52815-4dc9-4f7c-b572-2fc409a7b9a7 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:16.439310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e52815-4dc9-4f7c-b572-2fc409a7b9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:16.440058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e52815-4dc9-4f7c-b572-2fc409a7b9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:16.468396 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:16.501334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-d3738ae3-8f92-4dd4-8fbc-9a4dafa8f5f8 None None] MFA Rules not processed for user `f7b0dfb3d6514817aba4ab34fd972ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:16.502807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d3738ae3-8f92-4dd4-8fbc-9a4dafa8f5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:16.503608 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-f3e52815-4dc9-4f7c-b572-2fc409a7b9a7 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:16.509456 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f3e52815-4dc9-4f7c-b572-2fc409a7b9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:16.547258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 852/1708] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:16.548486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053222b-6436-45fc-97dc-eee87290168e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:16.548779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053222b-6436-45fc-97dc-eee87290168e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:16.555526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053222b-6436-45fc-97dc-eee87290168e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:16.589936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:16.654731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 857/1709] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:16.655584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3738ae3-8f92-4dd4-8fbc-9a4dafa8f5f8 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:16.671540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-d053222b-6436-45fc-97dc-eee87290168e None None] MFA Rules not processed for user `7ac98870c6524e879027d31300a07f28`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:16.677246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d053222b-6436-45fc-97dc-eee87290168e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:16.698883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3738ae3-8f92-4dd4-8fbc-9a4dafa8f5f8 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:16.723012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e358b-1a07-48bc-aea2-394de08ecd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:16.724312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e358b-1a07-48bc-aea2-394de08ecd9a admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:16.724885 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e358b-1a07-48bc-aea2-394de08ecd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:16.725191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e358b-1a07-48bc-aea2-394de08ecd9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:16.725931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e358b-1a07-48bc-aea2-394de08ecd9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:00:16.726258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e358b-1a07-48bc-aea2-394de08ecd9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:16.727951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e358b-1a07-48bc-aea2-394de08ecd9a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:16.758180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 858/1710] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:00:16.759076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e358b-1a07-48bc-aea2-394de08ecd9a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:16.764217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a34e358b-1a07-48bc-aea2-394de08ecd9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:16.797002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e358b-1a07-48bc-aea2-394de08ecd9a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:16.814250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ae842-6d31-4fdf-93e2-49e075a90c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:16.815240 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ae842-6d31-4fdf-93e2-49e075a90c1b service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:16.816367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ae842-6d31-4fdf-93e2-49e075a90c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:16.817518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ae842-6d31-4fdf-93e2-49e075a90c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:16.852101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ae842-6d31-4fdf-93e2-49e075a90c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:16.855165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ae842-6d31-4fdf-93e2-49e075a90c1b service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:16.909841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 859/1711] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:16.913623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ae842-6d31-4fdf-93e2-49e075a90c1b service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:16.914077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 853/1712] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:16.915102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053222b-6436-45fc-97dc-eee87290168e None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:16.924736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d053222b-6436-45fc-97dc-eee87290168e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:16.959128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ae842-6d31-4fdf-93e2-49e075a90c1b service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:16.978612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053222b-6436-45fc-97dc-eee87290168e None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:16.986365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30016542-5422-4dfe-978d-465f81b2887a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:16.987306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016542-5422-4dfe-978d-465f81b2887a admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:16.987739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016542-5422-4dfe-978d-465f81b2887a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:16.988164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016542-5422-4dfe-978d-465f81b2887a admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:16.989015 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30016542-5422-4dfe-978d-465f81b2887a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:16.990065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:16.991191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:16.992492 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30016542-5422-4dfe-978d-465f81b2887a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:16.992910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:16.993304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] PATH_INFO: `/v3/projects/d62a2c4bb19842ecb1bbfd1a40c96f4d` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:16.997417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62a2c4bb19842ecb1bbfd1a40c96f4d)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:17.000825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:17.063584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 860/1713] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:16 2022] POST /identity/v3/projects => generated 369 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:17.065863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30016542-5422-4dfe-978d-465f81b2887a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:17.068862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-30016542-5422-4dfe-978d-465f81b2887a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:17.165237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62a2c4bb19842ecb1bbfd1a40c96f4d', 'request_id': 'req-75f7b8c3-de59-45bc-86b8-13a3c383ee52'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:17.192182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62a2c4bb19842ecb1bbfd1a40c96f4d', 'request_id': 'req-75f7b8c3-de59-45bc-86b8-13a3c383ee52'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:17.210357 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62a2c4bb19842ecb1bbfd1a40c96f4d', 'request_id': 'req-75f7b8c3-de59-45bc-86b8-13a3c383ee52'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:17.265562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30016542-5422-4dfe-978d-465f81b2887a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:17.269325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] The token cache is being invalidate because project d62a2c4bb19842ecb1bbfd1a40c96f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:17.269325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:17.276678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:17.283389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 854/1714] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:16 2022] DELETE /identity/v3/projects/d62a2c4bb19842ecb1bbfd1a40c96f4d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:17.285687 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:17.300261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:17.435918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad3334-107e-44c5-b719-d24775a19499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:17.437438 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad3334-107e-44c5-b719-d24775a19499 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:17.437819 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad3334-107e-44c5-b719-d24775a19499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:17.438106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad3334-107e-44c5-b719-d24775a19499 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:17.438888 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad3334-107e-44c5-b719-d24775a19499 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:17.441299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad3334-107e-44c5-b719-d24775a19499 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:17.487278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 861/1715] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:17 2022] POST /identity/v3/projects => generated 403 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:17.492117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d49512-b5d7-4f18-924a-6a99bc09e9ea None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:17.492420 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d49512-b5d7-4f18-924a-6a99bc09e9ea None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:17.492926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d49512-b5d7-4f18-924a-6a99bc09e9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:17.536665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-71d49512-b5d7-4f18-924a-6a99bc09e9ea None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:17.538691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-71d49512-b5d7-4f18-924a-6a99bc09e9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:17.567462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:17.570758 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7b8c3-de59-45bc-86b8-13a3c383ee52 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:17.640686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:17.830316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6e3fe-4b11-4c0b-ba89-fc34266d7fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:17.831069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6e3fe-4b11-4c0b-ba89-fc34266d7fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:17.831326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6e3fe-4b11-4c0b-ba89-fc34266d7fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:17.831605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6e3fe-4b11-4c0b-ba89-fc34266d7fe4 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:17.832320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6e3fe-4b11-4c0b-ba89-fc34266d7fe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:17.839753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6e3fe-4b11-4c0b-ba89-fc34266d7fe4 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:17.902603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 862/1716] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:17.906868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d49512-b5d7-4f18-924a-6a99bc09e9ea None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:17.913069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-71d49512-b5d7-4f18-924a-6a99bc09e9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:17.925429 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 855/1717] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:17 2022] POST /identity/v3/users => generated 285 bytes in 641 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:17.928587 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7a564-d614-4870-af4e-c61d0db1faee None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:17.929183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7a564-d614-4870-af4e-c61d0db1faee None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:17.929665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7a564-d614-4870-af4e-c61d0db1faee None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:17.932559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 856/1718] 10.222.0.45 () {54 vars in 912 bytes} [Tue May 24 13:00:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:00:17.935187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f8e10-7b8e-45cd-976f-e12e244dd1d0 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:17.935538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f8e10-7b8e-45cd-976f-e12e244dd1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:17.936171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f8e10-7b8e-45cd-976f-e12e244dd1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:17.956215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d49512-b5d7-4f18-924a-6a99bc09e9ea None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:17.975171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd459f-90a6-4c01-b4cf-024297d9fc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:17.975893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd459f-90a6-4c01-b4cf-024297d9fc6d admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:17.976262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd459f-90a6-4c01-b4cf-024297d9fc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:17.976690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd459f-90a6-4c01-b4cf-024297d9fc6d admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:17.995755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd459f-90a6-4c01-b4cf-024297d9fc6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:18.000752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd459f-90a6-4c01-b4cf-024297d9fc6d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:18.034163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-3c0f8e10-7b8e-45cd-976f-e12e244dd1d0 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:18.038744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3c0f8e10-7b8e-45cd-976f-e12e244dd1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:18.071395 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 857/1719] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:18.075774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f8e10-7b8e-45cd-976f-e12e244dd1d0 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:18.088656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 863/1720] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:17 2022] POST /identity/v3/users => generated 308 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:18.091276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b9fb9-6f32-4bf8-af73-be63c7b5ac88 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:18.091583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b9fb9-6f32-4bf8-af73-be63c7b5ac88 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:18.092043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b9fb9-6f32-4bf8-af73-be63c7b5ac88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:18.111383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f8e10-7b8e-45cd-976f-e12e244dd1d0 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:18.135213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-2f9b9fb9-6f32-4bf8-af73-be63c7b5ac88 None None] MFA Rules not processed for user `bb111be6bc2946d3b50b8d99a89cf8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:18.136094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e42b59-797b-4ca1-87bf-f7957162b5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:18.136899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e42b59-797b-4ca1-87bf-f7957162b5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:18.137362 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e42b59-797b-4ca1-87bf-f7957162b5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:18.138018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2f9b9fb9-6f32-4bf8-af73-be63c7b5ac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:18.138412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e42b59-797b-4ca1-87bf-f7957162b5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:18.140136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e42b59-797b-4ca1-87bf-f7957162b5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:18.149533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e42b59-797b-4ca1-87bf-f7957162b5f3 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:18.180206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 858/1721] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:18.181316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e42b59-797b-4ca1-87bf-f7957162b5f3 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:18.224813 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e42b59-797b-4ca1-87bf-f7957162b5f3 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:18.239091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c9dd2-adbc-4fa3-b990-7e3654862622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:18.239967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c9dd2-adbc-4fa3-b990-7e3654862622 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:18.240370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c9dd2-adbc-4fa3-b990-7e3654862622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:18.240902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c9dd2-adbc-4fa3-b990-7e3654862622 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:18.241957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c9dd2-adbc-4fa3-b990-7e3654862622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:00:18.244482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c9dd2-adbc-4fa3-b990-7e3654862622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:18.247093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c9dd2-adbc-4fa3-b990-7e3654862622 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:18.271556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 864/1722] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 13:00:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:18.273147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b9fb9-6f32-4bf8-af73-be63c7b5ac88 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:18.295007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 859/1723] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:00:18.296028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c9dd2-adbc-4fa3-b990-7e3654862622 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:18.303010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b9fb9-6f32-4bf8-af73-be63c7b5ac88 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:18.321077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c9dd2-adbc-4fa3-b990-7e3654862622 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:18.329982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49fbdf-623d-4597-9b0d-0f7241425f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:18.331140 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49fbdf-623d-4597-9b0d-0f7241425f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:18.331407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49fbdf-623d-4597-9b0d-0f7241425f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:18.331751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49fbdf-623d-4597-9b0d-0f7241425f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:18.332600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49fbdf-623d-4597-9b0d-0f7241425f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:18.334441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49fbdf-623d-4597-9b0d-0f7241425f92 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:18.337771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc283719-96b0-4a46-baaf-eb3630f7d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:18.338512 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283719-96b0-4a46-baaf-eb3630f7d579 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:18.338797 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283719-96b0-4a46-baaf-eb3630f7d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:18.339074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283719-96b0-4a46-baaf-eb3630f7d579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:18.340268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc283719-96b0-4a46-baaf-eb3630f7d579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:18.347540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc283719-96b0-4a46-baaf-eb3630f7d579 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:18.366936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 860/1724] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:18.369824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49fbdf-623d-4597-9b0d-0f7241425f92 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:18.389696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 865/1725] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:18.395123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc283719-96b0-4a46-baaf-eb3630f7d579 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:18.404543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bc283719-96b0-4a46-baaf-eb3630f7d579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:18.408001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49fbdf-623d-4597-9b0d-0f7241425f92 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:18.438642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06449d-1bd1-49eb-9653-9c85f8aaa248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:18.439514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06449d-1bd1-49eb-9653-9c85f8aaa248 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:18.439851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06449d-1bd1-49eb-9653-9c85f8aaa248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:18.440240 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06449d-1bd1-49eb-9653-9c85f8aaa248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:18.441062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06449d-1bd1-49eb-9653-9c85f8aaa248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:18.443517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06449d-1bd1-49eb-9653-9c85f8aaa248 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:18.486241 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc283719-96b0-4a46-baaf-eb3630f7d579 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:18.507830 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 861/1726] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:18 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:18.507830 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06449d-1bd1-49eb-9653-9c85f8aaa248 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:18.510163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583aae7-b200-4542-9ee7-01e33610c9fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:18.513498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583aae7-b200-4542-9ee7-01e33610c9fe service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:18.514778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583aae7-b200-4542-9ee7-01e33610c9fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:18.515075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583aae7-b200-4542-9ee7-01e33610c9fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:18.542710 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583aae7-b200-4542-9ee7-01e33610c9fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:18.544893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583aae7-b200-4542-9ee7-01e33610c9fe service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:18.572534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 866/1727] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:18.573693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583aae7-b200-4542-9ee7-01e33610c9fe service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:18.613024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583aae7-b200-4542-9ee7-01e33610c9fe service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:18.669270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06449d-1bd1-49eb-9653-9c85f8aaa248 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:18.696816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43ce17-f333-42df-8495-46e006556f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:18.698576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43ce17-f333-42df-8495-46e006556f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:18.699022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43ce17-f333-42df-8495-46e006556f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:18.700738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43ce17-f333-42df-8495-46e006556f33 admin admin] PATH_INFO: `/v3/projects/e4fdc5e3659545f8b9e6291dab076fec/users/fa854f4c79254947a80027a42602609f/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:18.755500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43ce17-f333-42df-8495-46e006556f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fdc5e3659545f8b9e6291dab076fec, user_id=fa854f4c79254947a80027a42602609f, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:18.766630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43ce17-f333-42df-8495-46e006556f33 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:18.809809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 862/1728] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:18 2022] PUT /identity/v3/projects/e4fdc5e3659545f8b9e6291dab076fec/users/fa854f4c79254947a80027a42602609f/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:18.810919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43ce17-f333-42df-8495-46e006556f33 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:18.896024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ca965-773b-4f8b-b8b0-a3533bf1abfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:18.905583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ca965-773b-4f8b-b8b0-a3533bf1abfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:18.911498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ca965-773b-4f8b-b8b0-a3533bf1abfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:18.911915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ca965-773b-4f8b-b8b0-a3533bf1abfb admin admin] PATH_INFO: `/v3/projects/dc453ae3a5744be4b4caa6f5931d7dcc/users/48ed03ac198847dabf9b0d33664fdd2c/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:18.928604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:18.980124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ca965-773b-4f8b-b8b0-a3533bf1abfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc453ae3a5744be4b4caa6f5931d7dcc, user_id=48ed03ac198847dabf9b0d33664fdd2c, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:18.993292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ca965-773b-4f8b-b8b0-a3533bf1abfb admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:19.001447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43ce17-f333-42df-8495-46e006556f33 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:19.013877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2fb80-3abe-4df6-986b-0eac2d36f51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:19.015651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2fb80-3abe-4df6-986b-0eac2d36f51e admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:19.016621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2fb80-3abe-4df6-986b-0eac2d36f51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:19.016974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2fb80-3abe-4df6-986b-0eac2d36f51e admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:19.018980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2fb80-3abe-4df6-986b-0eac2d36f51e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:19.023257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2fb80-3abe-4df6-986b-0eac2d36f51e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:19.051905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 867/1729] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:18 2022] PUT /identity/v3/projects/dc453ae3a5744be4b4caa6f5931d7dcc/users/48ed03ac198847dabf9b0d33664fdd2c/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:19.055629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b4e87-18d6-40af-9a8f-bf680ed70a1e None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:19.058261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b4e87-18d6-40af-9a8f-bf680ed70a1e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:19.058261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b4e87-18d6-40af-9a8f-bf680ed70a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:19.071672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 863/1730] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:18 2022] POST /identity/v3/users => generated 292 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:19.073843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2fb80-3abe-4df6-986b-0eac2d36f51e admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:19.102529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2fb80-3abe-4df6-986b-0eac2d36f51e admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:19.118659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:19.164452 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-769b4e87-18d6-40af-9a8f-bf680ed70a1e None None] MFA Rules not processed for user `fa854f4c79254947a80027a42602609f`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:19.174018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-769b4e87-18d6-40af-9a8f-bf680ed70a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:19.319655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9d9ee-0810-4eca-bc6f-8c8900d69952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:19.322808 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9d9ee-0810-4eca-bc6f-8c8900d69952 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:19.323824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9d9ee-0810-4eca-bc6f-8c8900d69952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:19.328590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9d9ee-0810-4eca-bc6f-8c8900d69952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:19.329637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9d9ee-0810-4eca-bc6f-8c8900d69952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:19.331530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9d9ee-0810-4eca-bc6f-8c8900d69952 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:19.346917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 868/1731] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:19 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:19.350681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b4e87-18d6-40af-9a8f-bf680ed70a1e None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:19.361235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 864/1732] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:19.362455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9d9ee-0810-4eca-bc6f-8c8900d69952 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:19.385878 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b4e87-18d6-40af-9a8f-bf680ed70a1e None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:19.400808 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9d9ee-0810-4eca-bc6f-8c8900d69952 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:19.415021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe199b27-f859-4dc0-a15e-e341e935f63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:19.415776 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe199b27-f859-4dc0-a15e-e341e935f63f admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:19.416085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe199b27-f859-4dc0-a15e-e341e935f63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:19.416364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe199b27-f859-4dc0-a15e-e341e935f63f admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:19.417103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe199b27-f859-4dc0-a15e-e341e935f63f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:19.418791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe199b27-f859-4dc0-a15e-e341e935f63f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:19.435179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84653ed2-5319-433b-a358-98723dd87709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:19.435995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84653ed2-5319-433b-a358-98723dd87709 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:19.436291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84653ed2-5319-433b-a358-98723dd87709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:19.436576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84653ed2-5319-433b-a358-98723dd87709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:19.437290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84653ed2-5319-433b-a358-98723dd87709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:19.439490 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84653ed2-5319-433b-a358-98723dd87709 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:19.449592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 869/1733] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:19.451313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe199b27-f859-4dc0-a15e-e341e935f63f admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:19.465526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe199b27-f859-4dc0-a15e-e341e935f63f admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:19.477061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772d400-1794-4e70-a94a-387f34a47cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:19.479753 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772d400-1794-4e70-a94a-387f34a47cad admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:19.480290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772d400-1794-4e70-a94a-387f34a47cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:19.480965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772d400-1794-4e70-a94a-387f34a47cad admin admin] PATH_INFO: `/v3/projects/dc453ae3a5744be4b4caa6f5931d7dcc/users/48ed03ac198847dabf9b0d33664fdd2c/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:19.486578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772d400-1794-4e70-a94a-387f34a47cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc453ae3a5744be4b4caa6f5931d7dcc, user_id=48ed03ac198847dabf9b0d33664fdd2c, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:19.496531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772d400-1794-4e70-a94a-387f34a47cad admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:19.557802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 865/1734] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:19 2022] POST /identity/v3/projects => generated 367 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:19.565718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84653ed2-5319-433b-a358-98723dd87709 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:19.570354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 870/1735] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:19 2022] PUT /identity/v3/projects/dc453ae3a5744be4b4caa6f5931d7dcc/users/48ed03ac198847dabf9b0d33664fdd2c/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:19.572879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772d400-1794-4e70-a94a-387f34a47cad admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:19.838937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84653ed2-5319-433b-a358-98723dd87709 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:19.860651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28701cc-aa9e-4b80-a92f-2be2f5ebb917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:19.862409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28701cc-aa9e-4b80-a92f-2be2f5ebb917 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:19.862702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28701cc-aa9e-4b80-a92f-2be2f5ebb917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:19.865237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28701cc-aa9e-4b80-a92f-2be2f5ebb917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:19.866637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28701cc-aa9e-4b80-a92f-2be2f5ebb917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:19.871766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28701cc-aa9e-4b80-a92f-2be2f5ebb917 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:19.929348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 866/1736] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:19.933571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28701cc-aa9e-4b80-a92f-2be2f5ebb917 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:19.949607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772d400-1794-4e70-a94a-387f34a47cad admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:19.973656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28701cc-aa9e-4b80-a92f-2be2f5ebb917 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:19.975919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcfb6c-b029-4500-98c9-b01cff6a50b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:19.976631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcfb6c-b029-4500-98c9-b01cff6a50b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:19.976918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcfb6c-b029-4500-98c9-b01cff6a50b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:19.977486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcfb6c-b029-4500-98c9-b01cff6a50b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:19.978550 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bcfb6c-b029-4500-98c9-b01cff6a50b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:19.980080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22a7bb-5405-46ed-8603-041b51681ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:19.980387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bcfb6c-b029-4500-98c9-b01cff6a50b9 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:19.982102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22a7bb-5405-46ed-8603-041b51681ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:19.983014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22a7bb-5405-46ed-8603-041b51681ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:19.983491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22a7bb-5405-46ed-8603-041b51681ccb admin admin] PATH_INFO: `/v3/projects/70ebf2d73d3c4a7dbe3c8ffbd8224ab7/users/21828dbb27744a59b1fd9462e14fa0fb/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:20.021689 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22a7bb-5405-46ed-8603-041b51681ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=70ebf2d73d3c4a7dbe3c8ffbd8224ab7, user_id=21828dbb27744a59b1fd9462e14fa0fb, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:20.036307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22a7bb-5405-46ed-8603-041b51681ccb admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:20.045339 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 871/1737] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:19 2022] POST /identity/v3/users => generated 290 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:20.053079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcfb6c-b029-4500-98c9-b01cff6a50b9 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:20.076523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcfb6c-b029-4500-98c9-b01cff6a50b9 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:20.090726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 867/1738] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:19 2022] PUT /identity/v3/projects/70ebf2d73d3c4a7dbe3c8ffbd8224ab7/users/21828dbb27744a59b1fd9462e14fa0fb/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:20.103567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a0971-89bd-4fc4-a807-c510da501626 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:20.104009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a0971-89bd-4fc4-a807-c510da501626 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:20.104185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a0971-89bd-4fc4-a807-c510da501626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:20.112426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6d55b-f7da-42a2-8d50-14bf5e68d994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:20.113135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6d55b-f7da-42a2-8d50-14bf5e68d994 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:20.113423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6d55b-f7da-42a2-8d50-14bf5e68d994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:20.113704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6d55b-f7da-42a2-8d50-14bf5e68d994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:20.114490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6d55b-f7da-42a2-8d50-14bf5e68d994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:20.116134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6d55b-f7da-42a2-8d50-14bf5e68d994 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:20.133599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:20.149963 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 872/1739] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:20.163621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6d55b-f7da-42a2-8d50-14bf5e68d994 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:20.262953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6d55b-f7da-42a2-8d50-14bf5e68d994 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:20.264150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7e2a0971-89bd-4fc4-a807-c510da501626 None None] MFA Rules not processed for user `21828dbb27744a59b1fd9462e14fa0fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:20.270860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7e2a0971-89bd-4fc4-a807-c510da501626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:20.541259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:20.575086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 868/1740] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:20.583389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2dd9b-7fe2-47bd-9ef1-9bc11bf7ac65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:20.593892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2dd9b-7fe2-47bd-9ef1-9bc11bf7ac65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:20.595454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2dd9b-7fe2-47bd-9ef1-9bc11bf7ac65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:20.596852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2dd9b-7fe2-47bd-9ef1-9bc11bf7ac65 admin admin] PATH_INFO: `/v3/projects/4ef9b5ffbb504503833610d5e93c629d/users/2a4c6fba04004269b567b72d017c676b/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:20.598948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a0971-89bd-4fc4-a807-c510da501626 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:20.608915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7e2a0971-89bd-4fc4-a807-c510da501626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:20.692742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2dd9b-7fe2-47bd-9ef1-9bc11bf7ac65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef9b5ffbb504503833610d5e93c629d, user_id=2a4c6fba04004269b567b72d017c676b, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:20.706495 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2dd9b-7fe2-47bd-9ef1-9bc11bf7ac65 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:20.769587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 873/1741] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:20 2022] PUT /identity/v3/projects/4ef9b5ffbb504503833610d5e93c629d/users/2a4c6fba04004269b567b72d017c676b/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:20.774499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa667da-9f69-4939-98dd-b19ff6ecde34 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:20.775830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa667da-9f69-4939-98dd-b19ff6ecde34 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:20.777175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a0971-89bd-4fc4-a807-c510da501626 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:20.777629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa667da-9f69-4939-98dd-b19ff6ecde34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:20.871392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6fa667da-9f69-4939-98dd-b19ff6ecde34 None None] MFA Rules not processed for user `91a76f7af8a5467bbf567ca5b9db9ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:20.881846 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6fa667da-9f69-4939-98dd-b19ff6ecde34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:20.956879 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fc656-0301-40b4-b6c9-eee60d35f169 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:20.959662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fc656-0301-40b4-b6c9-eee60d35f169 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:20.962768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fc656-0301-40b4-b6c9-eee60d35f169 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:20.964819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fc656-0301-40b4-b6c9-eee60d35f169 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:21.173997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fc656-0301-40b4-b6c9-eee60d35f169 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:21.177117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fc656-0301-40b4-b6c9-eee60d35f169 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:21.227308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 874/1742] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:21.238216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa667da-9f69-4939-98dd-b19ff6ecde34 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:21.239487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 869/1743] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:21.252818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0e124-96ce-4e82-90d5-e7e58f1d6f86 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:21.254321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0e124-96ce-4e82-90d5-e7e58f1d6f86 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:21.256497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0e124-96ce-4e82-90d5-e7e58f1d6f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:21.278806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:21.324834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-95d0e124-96ce-4e82-90d5-e7e58f1d6f86 None None] MFA Rules not processed for user `91a76f7af8a5467bbf567ca5b9db9ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:21.328394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-95d0e124-96ce-4e82-90d5-e7e58f1d6f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:21.352744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 870/1744] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:21.354797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0e124-96ce-4e82-90d5-e7e58f1d6f86 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:21.489473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa667da-9f69-4939-98dd-b19ff6ecde34 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:21.508696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53c743-1547-4dd0-945d-d00e71e51446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:21.509528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53c743-1547-4dd0-945d-d00e71e51446 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:21.509921 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53c743-1547-4dd0-945d-d00e71e51446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:21.510174 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53c743-1547-4dd0-945d-d00e71e51446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:21.519922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53c743-1547-4dd0-945d-d00e71e51446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:21.536296 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53c743-1547-4dd0-945d-d00e71e51446 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:21.570014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 875/1745] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:21.573524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53c743-1547-4dd0-945d-d00e71e51446 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:21.576434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7b53c743-1547-4dd0-945d-d00e71e51446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:21.660261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0e124-96ce-4e82-90d5-e7e58f1d6f86 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:21.690073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4597b-a320-4202-aafc-74440d67778a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:21.691156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4597b-a320-4202-aafc-74440d67778a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:21.692786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4597b-a320-4202-aafc-74440d67778a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:21.693081 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4597b-a320-4202-aafc-74440d67778a admin admin] PATH_INFO: `/v3/users/21828dbb27744a59b1fd9462e14fa0fb` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:21.698789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4597b-a320-4202-aafc-74440d67778a admin admin] RBAC: Authorizing `identity:delete_user(user_id=21828dbb27744a59b1fd9462e14fa0fb)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:21.704701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4597b-a320-4202-aafc-74440d67778a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:21.883582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:21.889884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5fb4597b-a320-4202-aafc-74440d67778a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21828dbb27744a59b1fd9462e14fa0fb'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:21.917162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53c743-1547-4dd0-945d-d00e71e51446 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:21.932868 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5fb4597b-a320-4202-aafc-74440d67778a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21828dbb27744a59b1fd9462e14fa0fb'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:21.940211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a494c1-e0cb-40a8-b99a-19fcbb08dd25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:21.943844 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a494c1-e0cb-40a8-b99a-19fcbb08dd25 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:21.944834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a494c1-e0cb-40a8-b99a-19fcbb08dd25 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:21.945673 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a494c1-e0cb-40a8-b99a-19fcbb08dd25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:21.951599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5fb4597b-a320-4202-aafc-74440d67778a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21828dbb27744a59b1fd9462e14fa0fb'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:21.952302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5fb4597b-a320-4202-aafc-74440d67778a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21828dbb27744a59b1fd9462e14fa0fb'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:21.968237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5fb4597b-a320-4202-aafc-74440d67778a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21828dbb27744a59b1fd9462e14fa0fb'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:21.976219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a494c1-e0cb-40a8-b99a-19fcbb08dd25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:21.984114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 871/1746] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:21 2022] DELETE /identity/v3/users/21828dbb27744a59b1fd9462e14fa0fb => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:21.984114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3911ca1-d8d9-42de-9e95-90ead438e7e6 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:21.984114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3911ca1-d8d9-42de-9e95-90ead438e7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:21.984114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3911ca1-d8d9-42de-9e95-90ead438e7e6 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:21.987349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a494c1-e0cb-40a8-b99a-19fcbb08dd25 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:21.998068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 872/1747] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 13:00:21 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:00:21.998995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3911ca1-d8d9-42de-9e95-90ead438e7e6 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:22.006211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c3911ca1-d8d9-42de-9e95-90ead438e7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:22.029320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 876/1748] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:22.031212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd69920-89bf-464a-a1d2-620885ac6acd None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:22.031508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd69920-89bf-464a-a1d2-620885ac6acd None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:22.031821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd69920-89bf-464a-a1d2-620885ac6acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:22.073801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-2cd69920-89bf-464a-a1d2-620885ac6acd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:22.075274 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2cd69920-89bf-464a-a1d2-620885ac6acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:22.381372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3911ca1-d8d9-42de-9e95-90ead438e7e6 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:22.402839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8889c6-af25-4f1b-a9bc-433ba3cc5b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:22.404797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8889c6-af25-4f1b-a9bc-433ba3cc5b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:22.405238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8889c6-af25-4f1b-a9bc-433ba3cc5b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:22.406399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8889c6-af25-4f1b-a9bc-433ba3cc5b13 admin admin] PATH_INFO: `/v3/projects/4ef9b5ffbb504503833610d5e93c629d/users/2a4c6fba04004269b567b72d017c676b/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:22.412237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 877/1749] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:22.413903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8889c6-af25-4f1b-a9bc-433ba3cc5b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef9b5ffbb504503833610d5e93c629d, user_id=2a4c6fba04004269b567b72d017c676b, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:22.414898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a367a37-81a0-476e-8ee9-ad6b11651130 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:22.415729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a367a37-81a0-476e-8ee9-ad6b11651130 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:22.416347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a367a37-81a0-476e-8ee9-ad6b11651130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:22.418980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8889c6-af25-4f1b-a9bc-433ba3cc5b13 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:22.454319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 873/1750] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:21 2022] PUT /identity/v3/projects/4ef9b5ffbb504503833610d5e93c629d/users/2a4c6fba04004269b567b72d017c676b/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:22.455215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8889c6-af25-4f1b-a9bc-433ba3cc5b13 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:22.459871 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-dc8889c6-af25-4f1b-a9bc-433ba3cc5b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:22.474536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7a367a37-81a0-476e-8ee9-ad6b11651130 None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:22.488690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7a367a37-81a0-476e-8ee9-ad6b11651130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:22.565979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:22.805366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8889c6-af25-4f1b-a9bc-433ba3cc5b13 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:22.844581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695deb2-8840-412c-b7c4-3332411e9ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:22.845426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695deb2-8840-412c-b7c4-3332411e9ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:22.845794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695deb2-8840-412c-b7c4-3332411e9ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:22.846066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695deb2-8840-412c-b7c4-3332411e9ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:22.893144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 878/1751] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 13:00:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:22.896535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6efdf-3bfd-44ac-8715-2ae7d9ce5880 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:22.896907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6efdf-3bfd-44ac-8715-2ae7d9ce5880 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:22.897204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6efdf-3bfd-44ac-8715-2ae7d9ce5880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:22.974599 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-0ab6efdf-3bfd-44ac-8715-2ae7d9ce5880 None None] MFA Rules not processed for user `2a4c6fba04004269b567b72d017c676b`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:22.975915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0ab6efdf-3bfd-44ac-8715-2ae7d9ce5880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:23.073912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695deb2-8840-412c-b7c4-3332411e9ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:23.075300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695deb2-8840-412c-b7c4-3332411e9ae4 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:23.115681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 874/1752] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:23.120928 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2db7e6-bdc6-4237-bdde-1695d9fc3755 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:23.121268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2db7e6-bdc6-4237-bdde-1695d9fc3755 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:23.121690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2db7e6-bdc6-4237-bdde-1695d9fc3755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:23.199275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9b2db7e6-bdc6-4237-bdde-1695d9fc3755 None None] MFA Rules not processed for user `cb3d46a7a0ae40e9b157b7199b18de27`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:23.206461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9b2db7e6-bdc6-4237-bdde-1695d9fc3755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:23.262735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 879/1753] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:23.264859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6efdf-3bfd-44ac-8715-2ae7d9ce5880 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:23.273740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0ab6efdf-3bfd-44ac-8715-2ae7d9ce5880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:23.317746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6efdf-3bfd-44ac-8715-2ae7d9ce5880 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:23.349589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974942f-1868-4961-9616-f8c452690b63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:23.350686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974942f-1868-4961-9616-f8c452690b63 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:23.350942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974942f-1868-4961-9616-f8c452690b63 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:23.351220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974942f-1868-4961-9616-f8c452690b63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:23.362023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:23.385701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974942f-1868-4961-9616-f8c452690b63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:23.392073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974942f-1868-4961-9616-f8c452690b63 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:23.429039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 880/1754] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:23.437314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974942f-1868-4961-9616-f8c452690b63 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:23.441781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5974942f-1868-4961-9616-f8c452690b63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:23.445432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 875/1755] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:23.665002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974942f-1868-4961-9616-f8c452690b63 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:23.673676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0551f9-5e16-4f53-8bb9-37e9bb8834fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:23.674807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0551f9-5e16-4f53-8bb9-37e9bb8834fd service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:23.675101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0551f9-5e16-4f53-8bb9-37e9bb8834fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:23.675387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0551f9-5e16-4f53-8bb9-37e9bb8834fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:23.706132 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0551f9-5e16-4f53-8bb9-37e9bb8834fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:23.711762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0551f9-5e16-4f53-8bb9-37e9bb8834fd service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:23.746057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 881/1756] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:23 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:23.867338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2db7e6-bdc6-4237-bdde-1695d9fc3755 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:23.887751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2db7e6-bdc6-4237-bdde-1695d9fc3755 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:23.904619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24e736-b07e-4e0b-8abd-d07b91550e76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:23.906817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24e736-b07e-4e0b-8abd-d07b91550e76 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:23.908094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24e736-b07e-4e0b-8abd-d07b91550e76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:23.915805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24e736-b07e-4e0b-8abd-d07b91550e76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:23.924488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4a24e736-b07e-4e0b-8abd-d07b91550e76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:24.060783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24e736-b07e-4e0b-8abd-d07b91550e76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:24.065621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24e736-b07e-4e0b-8abd-d07b91550e76 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:24.094226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 876/1757] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:24.569494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0551f9-5e16-4f53-8bb9-37e9bb8834fd service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:24.573391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7d0551f9-5e16-4f53-8bb9-37e9bb8834fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:24.606805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0551f9-5e16-4f53-8bb9-37e9bb8834fd service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:24.627608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:24.628933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:24.629422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:24.629936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] PATH_INFO: `/v3/projects/70ebf2d73d3c4a7dbe3c8ffbd8224ab7` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:24.637462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70ebf2d73d3c4a7dbe3c8ffbd8224ab7)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:24.641142 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:24.821667 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70ebf2d73d3c4a7dbe3c8ffbd8224ab7', 'request_id': 'req-8ce75f5b-a3de-469f-a261-b9f7fb28b374'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:24.841222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70ebf2d73d3c4a7dbe3c8ffbd8224ab7', 'request_id': 'req-8ce75f5b-a3de-469f-a261-b9f7fb28b374'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:24.857456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70ebf2d73d3c4a7dbe3c8ffbd8224ab7', 'request_id': 'req-8ce75f5b-a3de-469f-a261-b9f7fb28b374'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:24.888106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] The token cache is being invalidate because project 70ebf2d73d3c4a7dbe3c8ffbd8224ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:24.897265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:24.899042 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-8ce75f5b-a3de-469f-a261-b9f7fb28b374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:24.903787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 882/1758] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:24 2022] DELETE /identity/v3/projects/70ebf2d73d3c4a7dbe3c8ffbd8224ab7 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:24.910415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788e6bc-bbdd-4f30-af3b-55d53287c6d5 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:24.910980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788e6bc-bbdd-4f30-af3b-55d53287c6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:24.910980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788e6bc-bbdd-4f30-af3b-55d53287c6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:24.928768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:24.974997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a788e6bc-bbdd-4f30-af3b-55d53287c6d5 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:24.995699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a788e6bc-bbdd-4f30-af3b-55d53287c6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:25.322651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 877/1759] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:25.331500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772088c-3a7d-4776-a51b-f18e834f6189 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:25.331911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772088c-3a7d-4776-a51b-f18e834f6189 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:25.332308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772088c-3a7d-4776-a51b-f18e834f6189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:25.413975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f772088c-3a7d-4776-a51b-f18e834f6189 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:25.416734 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f772088c-3a7d-4776-a51b-f18e834f6189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:25.450317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788e6bc-bbdd-4f30-af3b-55d53287c6d5 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:25.458170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 883/1760] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:25.468835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788e6bc-bbdd-4f30-af3b-55d53287c6d5 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:25.502706 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0dad7-fe2a-41ac-92ae-b2b3e0bf32b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:25.503417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0dad7-fe2a-41ac-92ae-b2b3e0bf32b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:25.503713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0dad7-fe2a-41ac-92ae-b2b3e0bf32b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:25.504003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0dad7-fe2a-41ac-92ae-b2b3e0bf32b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:25.505378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0dad7-fe2a-41ac-92ae-b2b3e0bf32b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:00:25.505699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0dad7-fe2a-41ac-92ae-b2b3e0bf32b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:25.510511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0dad7-fe2a-41ac-92ae-b2b3e0bf32b9 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:25.549801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 878/1761] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:00:25.558672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772088c-3a7d-4776-a51b-f18e834f6189 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:25.569463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772088c-3a7d-4776-a51b-f18e834f6189 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:25.594930 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97908734-7ac9-47e0-83c3-36ac88c3be23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:25.595774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97908734-7ac9-47e0-83c3-36ac88c3be23 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:25.596217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97908734-7ac9-47e0-83c3-36ac88c3be23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:25.609175 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97908734-7ac9-47e0-83c3-36ac88c3be23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:25.610715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97908734-7ac9-47e0-83c3-36ac88c3be23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:25.612944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97908734-7ac9-47e0-83c3-36ac88c3be23 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:25.730895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0dad7-fe2a-41ac-92ae-b2b3e0bf32b9 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:25.731666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 884/1762] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:25 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:25.747309 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0dad7-fe2a-41ac-92ae-b2b3e0bf32b9 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:25.972600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead838e9-7ea4-4f66-a28e-02f82154ce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:25.975603 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead838e9-7ea4-4f66-a28e-02f82154ce82 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:25.981295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead838e9-7ea4-4f66-a28e-02f82154ce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:25.981751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead838e9-7ea4-4f66-a28e-02f82154ce82 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:25.984145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead838e9-7ea4-4f66-a28e-02f82154ce82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:25.995241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead838e9-7ea4-4f66-a28e-02f82154ce82 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:26.036655 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf6a04-a4f8-4f43-a278-01bb80faae88 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:26.037247 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf6a04-a4f8-4f43-a278-01bb80faae88 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:26.037820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf6a04-a4f8-4f43-a278-01bb80faae88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:26.090892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 879/1763] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:25 2022] POST /identity/v3/users => generated 288 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:00:26.095543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead838e9-7ea4-4f66-a28e-02f82154ce82 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:26.096893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-0cbf6a04-a4f8-4f43-a278-01bb80faae88 None None] MFA Rules not processed for user `91a76f7af8a5467bbf567ca5b9db9ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:26.100216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0cbf6a04-a4f8-4f43-a278-01bb80faae88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:26.166730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead838e9-7ea4-4f66-a28e-02f82154ce82 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:26.185205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0327bee-b079-410d-a827-8048718edd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:26.188572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0327bee-b079-410d-a827-8048718edd3d admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:26.193290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0327bee-b079-410d-a827-8048718edd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:26.193770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0327bee-b079-410d-a827-8048718edd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:26.195009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0327bee-b079-410d-a827-8048718edd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:26.210769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0327bee-b079-410d-a827-8048718edd3d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:26.260310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 880/1764] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:26.277273 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0327bee-b079-410d-a827-8048718edd3d admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:26.312536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:26.317943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0327bee-b079-410d-a827-8048718edd3d admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:26.354962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b9fdd-30c7-44e5-90c0-1eb5a1b31143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:26.355684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b9fdd-30c7-44e5-90c0-1eb5a1b31143 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:26.355974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b9fdd-30c7-44e5-90c0-1eb5a1b31143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:26.356260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b9fdd-30c7-44e5-90c0-1eb5a1b31143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:26.357040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b9fdd-30c7-44e5-90c0-1eb5a1b31143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:26.368900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b9fdd-30c7-44e5-90c0-1eb5a1b31143 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:26.409218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 881/1765] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:26.410208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b9fdd-30c7-44e5-90c0-1eb5a1b31143 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:26.426348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 885/1766] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:26.431085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b9fdd-30c7-44e5-90c0-1eb5a1b31143 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:26.444943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f832-f47c-4eac-96ab-071ae8e57f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:26.446086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f832-f47c-4eac-96ab-071ae8e57f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:26.446570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f832-f47c-4eac-96ab-071ae8e57f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:26.447422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f832-f47c-4eac-96ab-071ae8e57f2b admin admin] PATH_INFO: `/v3/projects/baa960629c264974bcbd3f64c6f04504/users/6dfe4b5e48d445ef803faf8935a62aba/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:26.499041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2f832-f47c-4eac-96ab-071ae8e57f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa960629c264974bcbd3f64c6f04504, user_id=6dfe4b5e48d445ef803faf8935a62aba, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:26.501770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2f832-f47c-4eac-96ab-071ae8e57f2b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:26.543242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 882/1767] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:26 2022] PUT /identity/v3/projects/baa960629c264974bcbd3f64c6f04504/users/6dfe4b5e48d445ef803faf8935a62aba/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:26.553890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab3dd2-e2ef-470e-8b90-9e05b8654357 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:26.554962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab3dd2-e2ef-470e-8b90-9e05b8654357 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:26.556164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab3dd2-e2ef-470e-8b90-9e05b8654357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:26.662848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-05ab3dd2-e2ef-470e-8b90-9e05b8654357 None None] MFA Rules not processed for user `6dfe4b5e48d445ef803faf8935a62aba`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:26.667229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-05ab3dd2-e2ef-470e-8b90-9e05b8654357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:26.951075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 886/1768] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:26.951176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f832-f47c-4eac-96ab-071ae8e57f2b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:26.958217 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-88f2f832-f47c-4eac-96ab-071ae8e57f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:27.017951 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:27.041425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f832-f47c-4eac-96ab-071ae8e57f2b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:27.046754 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe02069-6f5d-4333-89e8-f0ec0afe35f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:27.048127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe02069-6f5d-4333-89e8-f0ec0afe35f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:27.048996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe02069-6f5d-4333-89e8-f0ec0afe35f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:27.049594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe02069-6f5d-4333-89e8-f0ec0afe35f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:27.065323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe02069-6f5d-4333-89e8-f0ec0afe35f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:27.067215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe02069-6f5d-4333-89e8-f0ec0afe35f3 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:27.105368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 883/1769] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:28.012950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bdfd7-8f79-4b39-b258-254747c2f0b0 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:28.013834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bdfd7-8f79-4b39-b258-254747c2f0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:28.014458 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bdfd7-8f79-4b39-b258-254747c2f0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:28.064099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-820bdfd7-8f79-4b39-b258-254747c2f0b0 None None] MFA Rules not processed for user `48ed03ac198847dabf9b0d33664fdd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:28.068515 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-820bdfd7-8f79-4b39-b258-254747c2f0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:28.255874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 887/1770] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:28 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:29.585559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe02069-6f5d-4333-89e8-f0ec0afe35f3 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:29.588476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-efe02069-6f5d-4333-89e8-f0ec0afe35f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:29.718103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe02069-6f5d-4333-89e8-f0ec0afe35f3 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:29.731851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438229fb-9c07-40a4-aee9-625bc88072be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:29.733144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438229fb-9c07-40a4-aee9-625bc88072be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:29.734067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438229fb-9c07-40a4-aee9-625bc88072be admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:29.734644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438229fb-9c07-40a4-aee9-625bc88072be admin admin] PATH_INFO: `/v3/users/fa854f4c79254947a80027a42602609f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:29.736112 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438229fb-9c07-40a4-aee9-625bc88072be admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa854f4c79254947a80027a42602609f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:29.739786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438229fb-9c07-40a4-aee9-625bc88072be admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:29.821079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-438229fb-9c07-40a4-aee9-625bc88072be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa854f4c79254947a80027a42602609f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:29.839745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-438229fb-9c07-40a4-aee9-625bc88072be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa854f4c79254947a80027a42602609f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:29.850105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-438229fb-9c07-40a4-aee9-625bc88072be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa854f4c79254947a80027a42602609f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:29.851272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-438229fb-9c07-40a4-aee9-625bc88072be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa854f4c79254947a80027a42602609f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:29.862045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-438229fb-9c07-40a4-aee9-625bc88072be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa854f4c79254947a80027a42602609f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:29.870070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 884/1771] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:29 2022] DELETE /identity/v3/users/fa854f4c79254947a80027a42602609f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:29.874974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368e8e7-9a0e-4b0e-b87b-e13c86acfc5d None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:29.875473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368e8e7-9a0e-4b0e-b87b-e13c86acfc5d None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:29.875584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368e8e7-9a0e-4b0e-b87b-e13c86acfc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:29.899628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8368e8e7-9a0e-4b0e-b87b-e13c86acfc5d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:29.901614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8368e8e7-9a0e-4b0e-b87b-e13c86acfc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:29.951149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5cc88-83c0-4ff1-96f3-bee369f2c3f3 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:29.951477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5cc88-83c0-4ff1-96f3-bee369f2c3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:29.952056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5cc88-83c0-4ff1-96f3-bee369f2c3f3 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:29.953469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 885/1772] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 13:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:00:29.966592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a75bfe-4f86-4397-b3fb-5146841d6e6a None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:29.966895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a75bfe-4f86-4397-b3fb-5146841d6e6a None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:29.967184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a75bfe-4f86-4397-b3fb-5146841d6e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:29.976816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 888/1773] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:29.994552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368e8e7-9a0e-4b0e-b87b-e13c86acfc5d None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:29.997849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8368e8e7-9a0e-4b0e-b87b-e13c86acfc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:30.019759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:30.107334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368e8e7-9a0e-4b0e-b87b-e13c86acfc5d None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:30.111760 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fe0eb-3894-46cc-be65-e25d38dd0dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:30.112842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fe0eb-3894-46cc-be65-e25d38dd0dae service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:30.114199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fe0eb-3894-46cc-be65-e25d38dd0dae service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:30.114954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fe0eb-3894-46cc-be65-e25d38dd0dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:30.128553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fe0eb-3894-46cc-be65-e25d38dd0dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:30.131028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fe0eb-3894-46cc-be65-e25d38dd0dae service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:30.148627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 889/1774] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:30.201388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-73a75bfe-4f86-4397-b3fb-5146841d6e6a None None] MFA Rules not processed for user `630f0527bd844de9be072c459dab96c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:30.202677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-73a75bfe-4f86-4397-b3fb-5146841d6e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:30.319614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:30.342826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 886/1775] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 13:00:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:30.374255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fe0eb-3894-46cc-be65-e25d38dd0dae service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:30.376031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-df4fe0eb-3894-46cc-be65-e25d38dd0dae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:30.405330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fe0eb-3894-46cc-be65-e25d38dd0dae service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:30.413622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76994bbf-59dd-46f1-af9a-83b6599b57ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:30.414319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76994bbf-59dd-46f1-af9a-83b6599b57ff service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:30.414930 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76994bbf-59dd-46f1-af9a-83b6599b57ff service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:30.415430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76994bbf-59dd-46f1-af9a-83b6599b57ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:30.439640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76994bbf-59dd-46f1-af9a-83b6599b57ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:30.441864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76994bbf-59dd-46f1-af9a-83b6599b57ff service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:30.470043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 890/1776] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:31.208949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eee5a-f236-416d-8f98-b629100b132b None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:31.209411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eee5a-f236-416d-8f98-b629100b132b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:31.209693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eee5a-f236-416d-8f98-b629100b132b None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:31.211325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 887/1777] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 13:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:00:31.228874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c6de-775e-40da-a6b4-4893506764a3 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:31.229404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c6de-775e-40da-a6b4-4893506764a3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:31.229525 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c6de-775e-40da-a6b4-4893506764a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:31.249189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6e89c6de-775e-40da-a6b4-4893506764a3 None None] MFA Rules not processed for user `630f0527bd844de9be072c459dab96c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:31.251135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6e89c6de-775e-40da-a6b4-4893506764a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:31.273641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 891/1778] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 13:00:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:00:31.301294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eee5a-f236-416d-8f98-b629100b132b None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:31.312084 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89c6de-775e-40da-a6b4-4893506764a3 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:31.313515 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6e89c6de-775e-40da-a6b4-4893506764a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:31.329557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eee5a-f236-416d-8f98-b629100b132b None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:31.330951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89c6de-775e-40da-a6b4-4893506764a3 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:31.335843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e92a5f-eaa9-43f7-9269-7e5190e235aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:31.336693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e92a5f-eaa9-43f7-9269-7e5190e235aa service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:31.337077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e92a5f-eaa9-43f7-9269-7e5190e235aa service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:31.338502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e92a5f-eaa9-43f7-9269-7e5190e235aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:31.342661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:31.343544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:31.345581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:31.346054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] PATH_INFO: `/v3/projects/e4fdc5e3659545f8b9e6291dab076fec` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:31.348179 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fdc5e3659545f8b9e6291dab076fec)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:31.349025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e92a5f-eaa9-43f7-9269-7e5190e235aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:31.350904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e92a5f-eaa9-43f7-9269-7e5190e235aa service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:31.351165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:31.379641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 888/1779] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:00:31.383559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e92a5f-eaa9-43f7-9269-7e5190e235aa service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:31.387733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-08e92a5f-eaa9-43f7-9269-7e5190e235aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:31.426363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e92a5f-eaa9-43f7-9269-7e5190e235aa service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:31.433142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:31.434181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:31.434757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:31.435052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] PATH_INFO: `/v3/users/7ac98870c6524e879027d31300a07f28` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:31.436279 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ac98870c6524e879027d31300a07f28)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:31.438544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:31.445679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fdc5e3659545f8b9e6291dab076fec', 'request_id': 'req-b166bfba-da0e-4a74-a634-3618d24cc399'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.455131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fdc5e3659545f8b9e6291dab076fec', 'request_id': 'req-b166bfba-da0e-4a74-a634-3618d24cc399'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.469345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fdc5e3659545f8b9e6291dab076fec', 'request_id': 'req-b166bfba-da0e-4a74-a634-3618d24cc399'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.481103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] The token cache is being invalidate because project e4fdc5e3659545f8b9e6291dab076fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:31.481511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.482369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.483956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 892/1780] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:31 2022] DELETE /identity/v3/projects/e4fdc5e3659545f8b9e6291dab076fec => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:31.487774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:31.489086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:31.501602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ac98870c6524e879027d31300a07f28'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.514523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ac98870c6524e879027d31300a07f28'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.523201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ac98870c6524e879027d31300a07f28'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.523845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ac98870c6524e879027d31300a07f28'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.533390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ac98870c6524e879027d31300a07f28'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.541338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 889/1781] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:31 2022] DELETE /identity/v3/users/7ac98870c6524e879027d31300a07f28 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:31.558475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166bfba-da0e-4a74-a634-3618d24cc399 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:31.580702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:31.626868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:31.627932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:31.628064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:31.628310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] PATH_INFO: `/v3/users/2a4c6fba04004269b567b72d017c676b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:31.629539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4c6fba04004269b567b72d017c676b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:31.631542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:31.691043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4c6fba04004269b567b72d017c676b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.700866 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4c6fba04004269b567b72d017c676b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.701622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4c6fba04004269b567b72d017c676b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.702378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4c6fba04004269b567b72d017c676b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.715428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4c6fba04004269b567b72d017c676b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.730332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 893/1782] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:31 2022] DELETE /identity/v3/users/2a4c6fba04004269b567b72d017c676b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:31.800756 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:31.802501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:31.820762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067a526-3d16-44fd-83a9-dd3244200193 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:31.888115 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:31.888998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:31.889278 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:31.889550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] PATH_INFO: `/v3/projects/f1e5a0af9bbb4755aa58218e2fb1fd66` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:31.890771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1e5a0af9bbb4755aa58218e2fb1fd66)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:31.892660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:31.940761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1e5a0af9bbb4755aa58218e2fb1fd66', 'request_id': 'req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.944547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:31.948921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1e5a0af9bbb4755aa58218e2fb1fd66', 'request_id': 'req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.956490 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1e5a0af9bbb4755aa58218e2fb1fd66', 'request_id': 'req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.969739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] The token cache is being invalidate because project f1e5a0af9bbb4755aa58218e2fb1fd66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:31.970028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.970699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:31.972443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 890/1783] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:31 2022] DELETE /identity/v3/projects/f1e5a0af9bbb4755aa58218e2fb1fd66 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:31.976417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:31.977941 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:31.990640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb1ea4-1b0d-4930-98d8-46d003d798d6 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.017462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.019057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:32.055070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1e15e-d777-4ccf-bf0a-fabe5836cf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.056175 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1e15e-d777-4ccf-bf0a-fabe5836cf92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.056302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1e15e-d777-4ccf-bf0a-fabe5836cf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.056581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1e15e-d777-4ccf-bf0a-fabe5836cf92 admin admin] PATH_INFO: `/v3/users/48ed03ac198847dabf9b0d33664fdd2c` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.057791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb1e15e-d777-4ccf-bf0a-fabe5836cf92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ed03ac198847dabf9b0d33664fdd2c)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.059814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb1e15e-d777-4ccf-bf0a-fabe5836cf92 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.107028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:32.107252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8e3e3-6fc4-4d49-a83b-f4b5348f23de admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.109008 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3bb1e15e-d777-4ccf-bf0a-fabe5836cf92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ed03ac198847dabf9b0d33664fdd2c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.112964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.113920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.114246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.114611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] PATH_INFO: `/v3/projects/4ef9b5ffbb504503833610d5e93c629d` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.116130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef9b5ffbb504503833610d5e93c629d)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.117264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3bb1e15e-d777-4ccf-bf0a-fabe5836cf92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ed03ac198847dabf9b0d33664fdd2c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.118315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.118961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3bb1e15e-d777-4ccf-bf0a-fabe5836cf92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ed03ac198847dabf9b0d33664fdd2c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.119800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3bb1e15e-d777-4ccf-bf0a-fabe5836cf92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ed03ac198847dabf9b0d33664fdd2c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.133487 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3bb1e15e-d777-4ccf-bf0a-fabe5836cf92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ed03ac198847dabf9b0d33664fdd2c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.146562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 894/1784] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:31 2022] DELETE /identity/v3/users/48ed03ac198847dabf9b0d33664fdd2c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:32.170160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef9b5ffbb504503833610d5e93c629d', 'request_id': 'req-f0e0d224-0820-46cc-b045-3f8580b6d5de'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.178221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef9b5ffbb504503833610d5e93c629d', 'request_id': 'req-f0e0d224-0820-46cc-b045-3f8580b6d5de'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.184352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef9b5ffbb504503833610d5e93c629d', 'request_id': 'req-f0e0d224-0820-46cc-b045-3f8580b6d5de'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.196198 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] The token cache is being invalidate because project 4ef9b5ffbb504503833610d5e93c629d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:32.196587 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.197326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f0e0d224-0820-46cc-b045-3f8580b6d5de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.199307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 891/1785] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:32 2022] DELETE /identity/v3/projects/4ef9b5ffbb504503833610d5e93c629d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:32.204248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd3e11-0dc7-4e9a-bfa5-1f6085a6f2aa None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.204570 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd3e11-0dc7-4e9a-bfa5-1f6085a6f2aa None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.204917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd3e11-0dc7-4e9a-bfa5-1f6085a6f2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.224149 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-dedd3e11-0dc7-4e9a-bfa5-1f6085a6f2aa None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:32.225460 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-dedd3e11-0dc7-4e9a-bfa5-1f6085a6f2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:32.292776 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 895/1786] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:32.302047 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda76fe-3499-47dd-a1bd-d9d8cd387d49 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.302601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda76fe-3499-47dd-a1bd-d9d8cd387d49 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.302692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda76fe-3499-47dd-a1bd-d9d8cd387d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.325727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd3e11-0dc7-4e9a-bfa5-1f6085a6f2aa None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.327214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-dedd3e11-0dc7-4e9a-bfa5-1f6085a6f2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:32.331584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-ebda76fe-3499-47dd-a1bd-d9d8cd387d49 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:32.332890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ebda76fe-3499-47dd-a1bd-d9d8cd387d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:32.341995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd3e11-0dc7-4e9a-bfa5-1f6085a6f2aa None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.347012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.347881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.348167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.348429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] PATH_INFO: `/v3/projects/dc453ae3a5744be4b4caa6f5931d7dcc` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.349650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc453ae3a5744be4b4caa6f5931d7dcc)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.351635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.362463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 892/1787] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:32.363502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda76fe-3499-47dd-a1bd-d9d8cd387d49 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.373502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda76fe-3499-47dd-a1bd-d9d8cd387d49 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.378458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b30b2-29b3-4048-aa3a-365815338c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.379146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b30b2-29b3-4048-aa3a-365815338c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.379461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b30b2-29b3-4048-aa3a-365815338c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.379834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b30b2-29b3-4048-aa3a-365815338c2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.380588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b30b2-29b3-4048-aa3a-365815338c2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:00:32.380936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b30b2-29b3-4048-aa3a-365815338c2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.383093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b30b2-29b3-4048-aa3a-365815338c2f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.396125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 893/1788] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:00:32.401756 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b30b2-29b3-4048-aa3a-365815338c2f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.409121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc453ae3a5744be4b4caa6f5931d7dcc', 'request_id': 'req-31c4bd17-351d-43ff-a33f-0baa4098678a'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.414523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b30b2-29b3-4048-aa3a-365815338c2f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.418227 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc453ae3a5744be4b4caa6f5931d7dcc', 'request_id': 'req-31c4bd17-351d-43ff-a33f-0baa4098678a'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.425222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc453ae3a5744be4b4caa6f5931d7dcc', 'request_id': 'req-31c4bd17-351d-43ff-a33f-0baa4098678a'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.440440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] The token cache is being invalidate because project dc453ae3a5744be4b4caa6f5931d7dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:32.440725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.441765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-31c4bd17-351d-43ff-a33f-0baa4098678a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:32.443781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 896/1789] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:32 2022] DELETE /identity/v3/projects/dc453ae3a5744be4b4caa6f5931d7dcc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:32.448849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ece129-c394-4501-b390-7bc006d951d2 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.449197 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ece129-c394-4501-b390-7bc006d951d2 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.449671 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ece129-c394-4501-b390-7bc006d951d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.468683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:32.469106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-93ece129-c394-4501-b390-7bc006d951d2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:32.470337 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-93ece129-c394-4501-b390-7bc006d951d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:32.495075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9a13b-f2a5-4256-a1b1-a6f7e3d6171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.495906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9a13b-f2a5-4256-a1b1-a6f7e3d6171b admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.496103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9a13b-f2a5-4256-a1b1-a6f7e3d6171b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.496409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9a13b-f2a5-4256-a1b1-a6f7e3d6171b admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.497685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9a13b-f2a5-4256-a1b1-a6f7e3d6171b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.499793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9a13b-f2a5-4256-a1b1-a6f7e3d6171b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.530060 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:32.559529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 894/1790] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:32 2022] POST /identity/v3/projects => generated 401 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:32.560441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 897/1791] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:32.561095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9a13b-f2a5-4256-a1b1-a6f7e3d6171b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.564960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-32f9a13b-f2a5-4256-a1b1-a6f7e3d6171b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:32.569948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a483aba-6a17-448b-8ff7-6d408001312f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.570253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a483aba-6a17-448b-8ff7-6d408001312f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.571078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a483aba-6a17-448b-8ff7-6d408001312f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.595782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8a483aba-6a17-448b-8ff7-6d408001312f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:32.595782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8a483aba-6a17-448b-8ff7-6d408001312f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:32.649550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9a13b-f2a5-4256-a1b1-a6f7e3d6171b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.654651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d098e1-3745-4f91-ac25-4823fb21e605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.655329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d098e1-3745-4f91-ac25-4823fb21e605 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.655614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d098e1-3745-4f91-ac25-4823fb21e605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.655894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d098e1-3745-4f91-ac25-4823fb21e605 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.656584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d098e1-3745-4f91-ac25-4823fb21e605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.658375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 898/1792] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:32.661905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d098e1-3745-4f91-ac25-4823fb21e605 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.662934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a483aba-6a17-448b-8ff7-6d408001312f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.671924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a483aba-6a17-448b-8ff7-6d408001312f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.676736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369189e0-9768-48a2-951c-10afadb8bf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.677381 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369189e0-9768-48a2-951c-10afadb8bf41 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.677663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369189e0-9768-48a2-951c-10afadb8bf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.677955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369189e0-9768-48a2-951c-10afadb8bf41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.678691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369189e0-9768-48a2-951c-10afadb8bf41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:00:32.679000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369189e0-9768-48a2-951c-10afadb8bf41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.680598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369189e0-9768-48a2-951c-10afadb8bf41 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.684981 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 895/1793] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:32 2022] POST /identity/v3/users => generated 301 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:32.686085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d098e1-3745-4f91-ac25-4823fb21e605 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.696317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 899/1794] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:00:32.696488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d098e1-3745-4f91-ac25-4823fb21e605 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.697411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369189e0-9768-48a2-951c-10afadb8bf41 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.702422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd290d-dbb6-4e87-87a8-b49d2c31c079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.703111 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd290d-dbb6-4e87-87a8-b49d2c31c079 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.704020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd290d-dbb6-4e87-87a8-b49d2c31c079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.704313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd290d-dbb6-4e87-87a8-b49d2c31c079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.705057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd290d-dbb6-4e87-87a8-b49d2c31c079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.706708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd290d-dbb6-4e87-87a8-b49d2c31c079 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.707513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369189e0-9768-48a2-951c-10afadb8bf41 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.713871 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572de3fb-b3ab-45fa-9b1f-59a29f3cf6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.714582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572de3fb-b3ab-45fa-9b1f-59a29f3cf6ab admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.715265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572de3fb-b3ab-45fa-9b1f-59a29f3cf6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.715494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572de3fb-b3ab-45fa-9b1f-59a29f3cf6ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.716278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572de3fb-b3ab-45fa-9b1f-59a29f3cf6ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.718729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572de3fb-b3ab-45fa-9b1f-59a29f3cf6ab admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.718843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 896/1795] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:00:32.722528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd290d-dbb6-4e87-87a8-b49d2c31c079 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.735542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd290d-dbb6-4e87-87a8-b49d2c31c079 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.743231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbb4b4-6995-46a9-8772-bab454dd6db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.744845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbb4b4-6995-46a9-8772-bab454dd6db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.745043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbb4b4-6995-46a9-8772-bab454dd6db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.745402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbb4b4-6995-46a9-8772-bab454dd6db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.746215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbb4b4-6995-46a9-8772-bab454dd6db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.748388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbb4b4-6995-46a9-8772-bab454dd6db8 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.751569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 900/1796] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:32 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:32.758343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572de3fb-b3ab-45fa-9b1f-59a29f3cf6ab admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.765801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 897/1797] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:00:32.769721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbb4b4-6995-46a9-8772-bab454dd6db8 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.783537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbb4b4-6995-46a9-8772-bab454dd6db8 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.843748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572de3fb-b3ab-45fa-9b1f-59a29f3cf6ab admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.849442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5759bb-3689-415e-86da-acd73f58148e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.850229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5759bb-3689-415e-86da-acd73f58148e admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.850647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5759bb-3689-415e-86da-acd73f58148e admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.851043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5759bb-3689-415e-86da-acd73f58148e admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.851831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5759bb-3689-415e-86da-acd73f58148e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.851944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67875d-052c-4b7c-8bcc-7f01890f9773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.852902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67875d-052c-4b7c-8bcc-7f01890f9773 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.853187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67875d-052c-4b7c-8bcc-7f01890f9773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.853466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67875d-052c-4b7c-8bcc-7f01890f9773 admin admin] PATH_INFO: `/v3/projects/3a1eb621a30640b0a29acc3fa12893f4/users/262da4b6c9c74da691011079e34d736d/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.854397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5759bb-3689-415e-86da-acd73f58148e admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.875732 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67875d-052c-4b7c-8bcc-7f01890f9773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1eb621a30640b0a29acc3fa12893f4, user_id=262da4b6c9c74da691011079e34d736d, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.878440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67875d-052c-4b7c-8bcc-7f01890f9773 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.886788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 901/1798] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:32 2022] POST /identity/v3/users => generated 283 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:32.888000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5759bb-3689-415e-86da-acd73f58148e admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:32.897626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 898/1799] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:32 2022] PUT /identity/v3/projects/3a1eb621a30640b0a29acc3fa12893f4/users/262da4b6c9c74da691011079e34d736d/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:32.906922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5759bb-3689-415e-86da-acd73f58148e admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:32.910983 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1414-da2d-4a46-b292-5230e1e74b45 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.911426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1414-da2d-4a46-b292-5230e1e74b45 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.913131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1414-da2d-4a46-b292-5230e1e74b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.948355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-ff0d1414-da2d-4a46-b292-5230e1e74b45 None None] MFA Rules not processed for user `262da4b6c9c74da691011079e34d736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:32.949568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ff0d1414-da2d-4a46-b292-5230e1e74b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:32.956158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:32.971751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:32.977954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd8b2-c72f-44cc-8855-292a1637028c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:32.978736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd8b2-c72f-44cc-8855-292a1637028c admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:32.979131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd8b2-c72f-44cc-8855-292a1637028c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:32.979809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd8b2-c72f-44cc-8855-292a1637028c admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:32.980644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbd8b2-c72f-44cc-8855-292a1637028c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:32.983127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbd8b2-c72f-44cc-8855-292a1637028c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:32.994661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 902/1800] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:32.998909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd8b2-c72f-44cc-8855-292a1637028c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.008775 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd8b2-c72f-44cc-8855-292a1637028c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.013942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be75da-9abf-4e12-b307-cf3655d19ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.014617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be75da-9abf-4e12-b307-cf3655d19ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.014906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be75da-9abf-4e12-b307-cf3655d19ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.015182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be75da-9abf-4e12-b307-cf3655d19ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.015929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be75da-9abf-4e12-b307-cf3655d19ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.017519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be75da-9abf-4e12-b307-cf3655d19ae8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.021653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 899/1801] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:00:33.026033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d1414-da2d-4a46-b292-5230e1e74b45 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.027984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 903/1802] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:00:33.031934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be75da-9abf-4e12-b307-cf3655d19ae8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.036263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d1414-da2d-4a46-b292-5230e1e74b45 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.041290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f04ce5-8032-46ca-be5f-b6ef2a7f4229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.041931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f04ce5-8032-46ca-be5f-b6ef2a7f4229 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.042216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f04ce5-8032-46ca-be5f-b6ef2a7f4229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.042329 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be75da-9abf-4e12-b307-cf3655d19ae8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.042531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f04ce5-8032-46ca-be5f-b6ef2a7f4229 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.043220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f04ce5-8032-46ca-be5f-b6ef2a7f4229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.045123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f04ce5-8032-46ca-be5f-b6ef2a7f4229 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.046917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91485da9-1984-48f0-80d4-4339ed5c60bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.047787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91485da9-1984-48f0-80d4-4339ed5c60bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.048076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91485da9-1984-48f0-80d4-4339ed5c60bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.048354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91485da9-1984-48f0-80d4-4339ed5c60bb admin admin] PATH_INFO: `/v3/projects/42a7f283766742ea8e19877d3dd86512/users/9e6c1d7a909142e6aae8f2558114f65b/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.066627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91485da9-1984-48f0-80d4-4339ed5c60bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a7f283766742ea8e19877d3dd86512, user_id=9e6c1d7a909142e6aae8f2558114f65b, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.069590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91485da9-1984-48f0-80d4-4339ed5c60bb admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.080242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 900/1803] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:33 2022] POST /identity/v3/projects => generated 401 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:00:33.081465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f04ce5-8032-46ca-be5f-b6ef2a7f4229 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.083598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 904/1804] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:33 2022] PUT /identity/v3/projects/42a7f283766742ea8e19877d3dd86512/users/9e6c1d7a909142e6aae8f2558114f65b/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:33.094389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2defdc-584d-47b3-926c-a49b488fe22b None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.094712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2defdc-584d-47b3-926c-a49b488fe22b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.095016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2defdc-584d-47b3-926c-a49b488fe22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.121837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-4b2defdc-584d-47b3-926c-a49b488fe22b None None] MFA Rules not processed for user `9e6c1d7a909142e6aae8f2558114f65b`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:33.123355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4b2defdc-584d-47b3-926c-a49b488fe22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:33.147040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f04ce5-8032-46ca-be5f-b6ef2a7f4229 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.188362 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 905/1805] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:00:33.192659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2defdc-584d-47b3-926c-a49b488fe22b None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.203867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792f903-06e0-404a-a721-21a3863cb1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.204550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792f903-06e0-404a-a721-21a3863cb1df admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.204836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792f903-06e0-404a-a721-21a3863cb1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.205113 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792f903-06e0-404a-a721-21a3863cb1df admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.205809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792f903-06e0-404a-a721-21a3863cb1df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.207793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792f903-06e0-404a-a721-21a3863cb1df admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.230902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 901/1806] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:33 2022] POST /identity/v3/users => generated 307 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:00:33.235075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792f903-06e0-404a-a721-21a3863cb1df admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.245449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792f903-06e0-404a-a721-21a3863cb1df admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.250626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf946d0-3445-4062-8a44-0c35ba3d5e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.251293 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf946d0-3445-4062-8a44-0c35ba3d5e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.251617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf946d0-3445-4062-8a44-0c35ba3d5e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.251894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf946d0-3445-4062-8a44-0c35ba3d5e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.252657 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf946d0-3445-4062-8a44-0c35ba3d5e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.254251 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf946d0-3445-4062-8a44-0c35ba3d5e31 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.264688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 902/1807] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:00:33.268783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf946d0-3445-4062-8a44-0c35ba3d5e31 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.271710 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2defdc-584d-47b3-926c-a49b488fe22b None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.276413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9edaca-9344-4f38-9efe-bfc015e0d48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.277078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9edaca-9344-4f38-9efe-bfc015e0d48a admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.277365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9edaca-9344-4f38-9efe-bfc015e0d48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.277643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9edaca-9344-4f38-9efe-bfc015e0d48a admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.278369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9edaca-9344-4f38-9efe-bfc015e0d48a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.278572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf946d0-3445-4062-8a44-0c35ba3d5e31 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.280325 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9edaca-9344-4f38-9efe-bfc015e0d48a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.283109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282159f-981f-4830-9d8f-e4234c66411d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.283973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282159f-981f-4830-9d8f-e4234c66411d admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.284258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282159f-981f-4830-9d8f-e4234c66411d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.284540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282159f-981f-4830-9d8f-e4234c66411d admin admin] PATH_INFO: `/v3/projects/b738714e83ff457e940b51fcced26578/users/65dd9f32cf98448abae4cd380ca24fae/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.300055 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282159f-981f-4830-9d8f-e4234c66411d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b738714e83ff457e940b51fcced26578, user_id=65dd9f32cf98448abae4cd380ca24fae, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.302840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282159f-981f-4830-9d8f-e4234c66411d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.311004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 906/1808] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:33 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:00:33.312200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:33.315261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9edaca-9344-4f38-9efe-bfc015e0d48a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.318748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 903/1809] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:33 2022] PUT /identity/v3/projects/b738714e83ff457e940b51fcced26578/users/65dd9f32cf98448abae4cd380ca24fae/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:33.323044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282159f-981f-4830-9d8f-e4234c66411d admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.337328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:33.388626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282159f-981f-4830-9d8f-e4234c66411d admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.393390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e9f56-3ff9-4c27-8d84-3e417d751da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.394109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9f56-3ff9-4c27-8d84-3e417d751da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.394441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9f56-3ff9-4c27-8d84-3e417d751da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.394721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9f56-3ff9-4c27-8d84-3e417d751da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.395490 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e9f56-3ff9-4c27-8d84-3e417d751da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.397122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e9f56-3ff9-4c27-8d84-3e417d751da2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.399510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9edaca-9344-4f38-9efe-bfc015e0d48a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.405205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b215150-0e4a-4410-bf9f-efdf29e74349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.405969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215150-0e4a-4410-bf9f-efdf29e74349 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.406258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215150-0e4a-4410-bf9f-efdf29e74349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.406590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215150-0e4a-4410-bf9f-efdf29e74349 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.407344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b215150-0e4a-4410-bf9f-efdf29e74349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.410381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 904/1810] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:00:33.410557 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b215150-0e4a-4410-bf9f-efdf29e74349 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.413747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e9f56-3ff9-4c27-8d84-3e417d751da2 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.425758 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e9f56-3ff9-4c27-8d84-3e417d751da2 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.431559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9ee5c-5e53-448f-bb93-0207d9d27ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.432534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9ee5c-5e53-448f-bb93-0207d9d27ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.432863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9ee5c-5e53-448f-bb93-0207d9d27ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.433183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9ee5c-5e53-448f-bb93-0207d9d27ec7 admin admin] PATH_INFO: `/v3/projects/b738714e83ff457e940b51fcced26578/users/65dd9f32cf98448abae4cd380ca24fae/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.433515 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 907/1811] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:33 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:00:33.435567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9ee5c-5e53-448f-bb93-0207d9d27ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b738714e83ff457e940b51fcced26578, user_id=65dd9f32cf98448abae4cd380ca24fae, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.437151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b215150-0e4a-4410-bf9f-efdf29e74349 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.439114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9ee5c-5e53-448f-bb93-0207d9d27ec7 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.449885 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b215150-0e4a-4410-bf9f-efdf29e74349 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.451681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 905/1812] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:33 2022] PUT /identity/v3/projects/b738714e83ff457e940b51fcced26578/users/65dd9f32cf98448abae4cd380ca24fae/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:33.464267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d448da4-aec4-4408-adbe-739f390c6cd5 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.464676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d448da4-aec4-4408-adbe-739f390c6cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.465229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d448da4-aec4-4408-adbe-739f390c6cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.496022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1d448da4-aec4-4408-adbe-739f390c6cd5 None None] MFA Rules not processed for user `65dd9f32cf98448abae4cd380ca24fae`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:33.497359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1d448da4-aec4-4408-adbe-739f390c6cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:33.523972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21228de-1fb5-4bf9-918e-7096dfa28711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.525421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21228de-1fb5-4bf9-918e-7096dfa28711 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.525852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21228de-1fb5-4bf9-918e-7096dfa28711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.526322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21228de-1fb5-4bf9-918e-7096dfa28711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.527895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21228de-1fb5-4bf9-918e-7096dfa28711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.530717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21228de-1fb5-4bf9-918e-7096dfa28711 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.546099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 908/1813] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:33.549947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21228de-1fb5-4bf9-918e-7096dfa28711 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.561900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21228de-1fb5-4bf9-918e-7096dfa28711 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.568669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c92da-9500-4189-a32d-977aae46aeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.569569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c92da-9500-4189-a32d-977aae46aeab admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.569832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c92da-9500-4189-a32d-977aae46aeab admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.570115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c92da-9500-4189-a32d-977aae46aeab admin admin] PATH_INFO: `/v3/projects/26526de3616b483ca21ec5a05cfa6a9d/users/7a3f6e3648a040c186162058be281a68/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.591677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c92da-9500-4189-a32d-977aae46aeab admin admin] RBAC: Authorizing `identity:create_grant(project_id=26526de3616b483ca21ec5a05cfa6a9d, user_id=7a3f6e3648a040c186162058be281a68, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.594975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 906/1814] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:33 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:00:33.598993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c92da-9500-4189-a32d-977aae46aeab admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.613717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d448da4-aec4-4408-adbe-739f390c6cd5 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.614578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 909/1815] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:33 2022] PUT /identity/v3/projects/26526de3616b483ca21ec5a05cfa6a9d/users/7a3f6e3648a040c186162058be281a68/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:33.617365 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1d448da4-aec4-4408-adbe-739f390c6cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:33.619649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c92da-9500-4189-a32d-977aae46aeab admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.688777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d448da4-aec4-4408-adbe-739f390c6cd5 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.705454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c92da-9500-4189-a32d-977aae46aeab admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.710591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494eb02-b39f-4017-81f1-10fb25572ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.711269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494eb02-b39f-4017-81f1-10fb25572ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.711558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494eb02-b39f-4017-81f1-10fb25572ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.711838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494eb02-b39f-4017-81f1-10fb25572ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.713467 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494eb02-b39f-4017-81f1-10fb25572ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.715674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494eb02-b39f-4017-81f1-10fb25572ba6 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.716463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:33.725267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 910/1816] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:00:33.731027 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494eb02-b39f-4017-81f1-10fb25572ba6 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.740783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494eb02-b39f-4017-81f1-10fb25572ba6 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.746158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c34fc-ddb4-4ce4-b40c-8647e103c4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.747089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c34fc-ddb4-4ce4-b40c-8647e103c4dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.747375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c34fc-ddb4-4ce4-b40c-8647e103c4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.748585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c34fc-ddb4-4ce4-b40c-8647e103c4dc admin admin] PATH_INFO: `/v3/projects/26526de3616b483ca21ec5a05cfa6a9d/users/7a3f6e3648a040c186162058be281a68/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.750221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c34fc-ddb4-4ce4-b40c-8647e103c4dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=26526de3616b483ca21ec5a05cfa6a9d, user_id=7a3f6e3648a040c186162058be281a68, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.752748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c34fc-ddb4-4ce4-b40c-8647e103c4dc admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.757406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f37ac-2922-4e27-95b1-6a47b0f0dd12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.758521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f37ac-2922-4e27-95b1-6a47b0f0dd12 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.758746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f37ac-2922-4e27-95b1-6a47b0f0dd12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.759134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f37ac-2922-4e27-95b1-6a47b0f0dd12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.768147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 911/1817] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:33 2022] PUT /identity/v3/projects/26526de3616b483ca21ec5a05cfa6a9d/users/7a3f6e3648a040c186162058be281a68/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:33.777665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6327e6-60de-4d21-bd4b-3308ab47678c None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.777960 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6327e6-60de-4d21-bd4b-3308ab47678c None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.778250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6327e6-60de-4d21-bd4b-3308ab47678c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.784164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:33.806665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9e6327e6-60de-4d21-bd4b-3308ab47678c None None] MFA Rules not processed for user `7a3f6e3648a040c186162058be281a68`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:33.807948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9e6327e6-60de-4d21-bd4b-3308ab47678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:33.832649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f37ac-2922-4e27-95b1-6a47b0f0dd12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.834523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f37ac-2922-4e27-95b1-6a47b0f0dd12 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.846134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 907/1818] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:00:33.880707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 912/1819] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:33.892374 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f37ac-2922-4e27-95b1-6a47b0f0dd12 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.893788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c29f37ac-2922-4e27-95b1-6a47b0f0dd12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:33.906996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f37ac-2922-4e27-95b1-6a47b0f0dd12 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.936812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6327e6-60de-4d21-bd4b-3308ab47678c None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:33.964624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18359bcf-8f6b-483b-a7bd-4460245b57b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:33.965301 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18359bcf-8f6b-483b-a7bd-4460245b57b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:33.965663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18359bcf-8f6b-483b-a7bd-4460245b57b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:33.966013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18359bcf-8f6b-483b-a7bd-4460245b57b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:33.975454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18359bcf-8f6b-483b-a7bd-4460245b57b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:33.976888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18359bcf-8f6b-483b-a7bd-4460245b57b3 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:33.987266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 908/1820] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:33.995018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6327e6-60de-4d21-bd4b-3308ab47678c None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:33.999841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7986c7-8549-4a9f-a7d2-7d1d7596d58e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:34.000528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7986c7-8549-4a9f-a7d2-7d1d7596d58e service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:34.000822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7986c7-8549-4a9f-a7d2-7d1d7596d58e service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:34.001210 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7986c7-8549-4a9f-a7d2-7d1d7596d58e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:34.068112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7986c7-8549-4a9f-a7d2-7d1d7596d58e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:34.069125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7986c7-8549-4a9f-a7d2-7d1d7596d58e service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:34.078774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 913/1821] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:00:34.467394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18359bcf-8f6b-483b-a7bd-4460245b57b3 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:34.531068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18359bcf-8f6b-483b-a7bd-4460245b57b3 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:34.535646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:34.536521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:34.536835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:34.537108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] PATH_INFO: `/v3/users/9e6c1d7a909142e6aae8f2558114f65b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:34.538299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e6c1d7a909142e6aae8f2558114f65b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:34.540770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:34.556509 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:34.585520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e6c1d7a909142e6aae8f2558114f65b'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:34.597634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e6c1d7a909142e6aae8f2558114f65b'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:34.606304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e6c1d7a909142e6aae8f2558114f65b'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:34.606939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e6c1d7a909142e6aae8f2558114f65b'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:34.613824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e6c1d7a909142e6aae8f2558114f65b'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:34.620164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 909/1822] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:34 2022] DELETE /identity/v3/users/9e6c1d7a909142e6aae8f2558114f65b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:34.624820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a268f-81d7-438b-b074-fe8586a8f2e4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:34.625283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a268f-81d7-438b-b074-fe8586a8f2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:34.625390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a268f-81d7-438b-b074-fe8586a8f2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:34.641813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-976a268f-81d7-438b-b074-fe8586a8f2e4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:34.643038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-976a268f-81d7-438b-b074-fe8586a8f2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:34.680110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:34.701668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 914/1823] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:34.711036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:34.712439 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:34.769347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13a0b0-f5e9-4030-8315-eccae76d34db admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:34.772864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6b14b-6228-4e7c-b677-1b5f7e3371d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:34.773516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6b14b-6228-4e7c-b677-1b5f7e3371d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:34.773798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6b14b-6228-4e7c-b677-1b5f7e3371d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:34.774073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6b14b-6228-4e7c-b677-1b5f7e3371d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:34.783375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6b14b-6228-4e7c-b677-1b5f7e3371d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:34.784820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6b14b-6228-4e7c-b677-1b5f7e3371d7 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:34.795196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 910/1824] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:35.085948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a268f-81d7-438b-b074-fe8586a8f2e4 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:35.087916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-976a268f-81d7-438b-b074-fe8586a8f2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:35.099933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a268f-81d7-438b-b074-fe8586a8f2e4 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:35.105846 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:35.106750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:35.107034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:35.107313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] PATH_INFO: `/v3/projects/42a7f283766742ea8e19877d3dd86512` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:35.108512 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a7f283766742ea8e19877d3dd86512)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:35.110479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:35.158639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a7f283766742ea8e19877d3dd86512', 'request_id': 'req-5df0841b-98ea-457e-8e2f-cf943f827307'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.165671 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a7f283766742ea8e19877d3dd86512', 'request_id': 'req-5df0841b-98ea-457e-8e2f-cf943f827307'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.171311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a7f283766742ea8e19877d3dd86512', 'request_id': 'req-5df0841b-98ea-457e-8e2f-cf943f827307'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.183306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] The token cache is being invalidate because project 42a7f283766742ea8e19877d3dd86512 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:35.183592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.184242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.185894 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 915/1825] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:35 2022] DELETE /identity/v3/projects/42a7f283766742ea8e19877d3dd86512 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:35.189736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6b14b-6228-4e7c-b677-1b5f7e3371d7 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:35.200231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6b14b-6228-4e7c-b677-1b5f7e3371d7 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:35.272454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bac4bd-3b17-4a99-916e-e962ff48bb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:35.273799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bac4bd-3b17-4a99-916e-e962ff48bb29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:35.274238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bac4bd-3b17-4a99-916e-e962ff48bb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:35.274743 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bac4bd-3b17-4a99-916e-e962ff48bb29 admin admin] PATH_INFO: `/v3/users/7a3f6e3648a040c186162058be281a68` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:35.276545 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bac4bd-3b17-4a99-916e-e962ff48bb29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3f6e3648a040c186162058be281a68)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:35.279781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bac4bd-3b17-4a99-916e-e962ff48bb29 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:35.333633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:35.341089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-26bac4bd-3b17-4a99-916e-e962ff48bb29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3f6e3648a040c186162058be281a68'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.354403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-26bac4bd-3b17-4a99-916e-e962ff48bb29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3f6e3648a040c186162058be281a68'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.362439 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-26bac4bd-3b17-4a99-916e-e962ff48bb29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3f6e3648a040c186162058be281a68'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.363782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-26bac4bd-3b17-4a99-916e-e962ff48bb29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3f6e3648a040c186162058be281a68'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.371604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-26bac4bd-3b17-4a99-916e-e962ff48bb29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3f6e3648a040c186162058be281a68'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.377908 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 911/1826] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:35 2022] DELETE /identity/v3/users/7a3f6e3648a040c186162058be281a68 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:35.651778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:35.653235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:35.667781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0841b-98ea-457e-8e2f-cf943f827307 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:35.723230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:35.724103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:35.724388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:35.724664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] PATH_INFO: `/v3/projects/26526de3616b483ca21ec5a05cfa6a9d` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:35.725832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26526de3616b483ca21ec5a05cfa6a9d)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:35.728344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:35.772467 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26526de3616b483ca21ec5a05cfa6a9d', 'request_id': 'req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.779478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26526de3616b483ca21ec5a05cfa6a9d', 'request_id': 'req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.785308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26526de3616b483ca21ec5a05cfa6a9d', 'request_id': 'req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.787158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:35.799393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] The token cache is being invalidate because project 26526de3616b483ca21ec5a05cfa6a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:35.799552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.800321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-00f0f8ee-832f-4ad1-81d8-541eb2f22e80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:35.802135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 916/1827] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:35 2022] DELETE /identity/v3/projects/26526de3616b483ca21ec5a05cfa6a9d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:35.807264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaded7-53fd-4fb7-8f55-d108f801db14 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:35.807751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaded7-53fd-4fb7-8f55-d108f801db14 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:35.808206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaded7-53fd-4fb7-8f55-d108f801db14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:35.826574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-cceaded7-53fd-4fb7-8f55-d108f801db14 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:35.827694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-cceaded7-53fd-4fb7-8f55-d108f801db14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:35.887738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 912/1828] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:35.896325 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11013365-d003-4cb9-8bb3-c6defcc50378 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:35.896617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11013365-d003-4cb9-8bb3-c6defcc50378 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:35.896898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11013365-d003-4cb9-8bb3-c6defcc50378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:35.916030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-11013365-d003-4cb9-8bb3-c6defcc50378 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:35.917220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-11013365-d003-4cb9-8bb3-c6defcc50378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:35.925314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 917/1829] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:35.929344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceaded7-53fd-4fb7-8f55-d108f801db14 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:35.938919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceaded7-53fd-4fb7-8f55-d108f801db14 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:35.943518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b663e4-dd10-4296-bcd4-151e8df64329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:35.944167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b663e4-dd10-4296-bcd4-151e8df64329 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:35.944455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b663e4-dd10-4296-bcd4-151e8df64329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:35.944734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b663e4-dd10-4296-bcd4-151e8df64329 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:35.945455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b663e4-dd10-4296-bcd4-151e8df64329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:00:35.945765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b663e4-dd10-4296-bcd4-151e8df64329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:35.947384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b663e4-dd10-4296-bcd4-151e8df64329 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:35.958674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 913/1830] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:00:35.963146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11013365-d003-4cb9-8bb3-c6defcc50378 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:35.972161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11013365-d003-4cb9-8bb3-c6defcc50378 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:35.976751 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f65f62-9767-4a3d-8b78-b87bba637505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:35.977392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f65f62-9767-4a3d-8b78-b87bba637505 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:35.977721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f65f62-9767-4a3d-8b78-b87bba637505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:35.978536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f65f62-9767-4a3d-8b78-b87bba637505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:35.979220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f65f62-9767-4a3d-8b78-b87bba637505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:35.981125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f65f62-9767-4a3d-8b78-b87bba637505 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:36.007200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 918/1831] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:35 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:36.011051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b663e4-dd10-4296-bcd4-151e8df64329 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:36.021090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b663e4-dd10-4296-bcd4-151e8df64329 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:36.083344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef2564-c8ab-44e0-b5bf-c11dddd9d5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:36.084025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef2564-c8ab-44e0-b5bf-c11dddd9d5df admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.084307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef2564-c8ab-44e0-b5bf-c11dddd9d5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.084581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef2564-c8ab-44e0-b5bf-c11dddd9d5df admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.085282 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef2564-c8ab-44e0-b5bf-c11dddd9d5df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:36.087859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef2564-c8ab-44e0-b5bf-c11dddd9d5df admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:36.110529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 914/1832] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:36 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:36.115082 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f65f62-9767-4a3d-8b78-b87bba637505 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:36.135895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f65f62-9767-4a3d-8b78-b87bba637505 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:36.141087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbf775-eb12-4c37-9fc7-692ca527e296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:36.142911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbf775-eb12-4c37-9fc7-692ca527e296 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.142911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbf775-eb12-4c37-9fc7-692ca527e296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.142911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbf775-eb12-4c37-9fc7-692ca527e296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.143238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbf775-eb12-4c37-9fc7-692ca527e296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:36.144872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbf775-eb12-4c37-9fc7-692ca527e296 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:36.157962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 919/1833] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:36.161799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef2564-c8ab-44e0-b5bf-c11dddd9d5df admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:36.172202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef2564-c8ab-44e0-b5bf-c11dddd9d5df admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:36.177008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc774d5-d134-4d9e-8a56-2af50bf41e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:36.177676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc774d5-d134-4d9e-8a56-2af50bf41e6d admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.177963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc774d5-d134-4d9e-8a56-2af50bf41e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.178239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc774d5-d134-4d9e-8a56-2af50bf41e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.179070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc774d5-d134-4d9e-8a56-2af50bf41e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:36.180766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc774d5-d134-4d9e-8a56-2af50bf41e6d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:36.192470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 915/1834] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:36.196971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbf775-eb12-4c37-9fc7-692ca527e296 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:36.206749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbf775-eb12-4c37-9fc7-692ca527e296 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:36.211254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e2fdd-9710-4dd7-aa11-912d56da9902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:36.212677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e2fdd-9710-4dd7-aa11-912d56da9902 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.212976 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e2fdd-9710-4dd7-aa11-912d56da9902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.213253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e2fdd-9710-4dd7-aa11-912d56da9902 admin admin] PATH_INFO: `/v3/projects/7cd9be447af844249a5a696ceeaa4d23/users/1a15083ca15a4a32be4bc439655102b2/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.227205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e2fdd-9710-4dd7-aa11-912d56da9902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cd9be447af844249a5a696ceeaa4d23, user_id=1a15083ca15a4a32be4bc439655102b2, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:36.229949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e2fdd-9710-4dd7-aa11-912d56da9902 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:36.242423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 920/1835] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:36 2022] PUT /identity/v3/projects/7cd9be447af844249a5a696ceeaa4d23/users/1a15083ca15a4a32be4bc439655102b2/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:36.253477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23150399-d12b-4c40-bd92-c8c88456e22e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.253846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23150399-d12b-4c40-bd92-c8c88456e22e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.254154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23150399-d12b-4c40-bd92-c8c88456e22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.281087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-23150399-d12b-4c40-bd92-c8c88456e22e None None] MFA Rules not processed for user `1a15083ca15a4a32be4bc439655102b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:36.282344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-23150399-d12b-4c40-bd92-c8c88456e22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:36.293683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:36.354440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 916/1836] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:36.358611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e2fdd-9710-4dd7-aa11-912d56da9902 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:36.424191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e2fdd-9710-4dd7-aa11-912d56da9902 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:36.429131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8017746-7182-4814-b9cb-d951d4651678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:36.429803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8017746-7182-4814-b9cb-d951d4651678 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.430159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8017746-7182-4814-b9cb-d951d4651678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.430482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8017746-7182-4814-b9cb-d951d4651678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.431720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8017746-7182-4814-b9cb-d951d4651678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:36.433651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8017746-7182-4814-b9cb-d951d4651678 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:36.462615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 921/1837] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:36 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:36.463677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23150399-d12b-4c40-bd92-c8c88456e22e None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:36.474480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23150399-d12b-4c40-bd92-c8c88456e22e None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:36.531124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f2ace-94c3-4559-b98e-5cdd91f270e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:36.531798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f2ace-94c3-4559-b98e-5cdd91f270e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.532086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f2ace-94c3-4559-b98e-5cdd91f270e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.532360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f2ace-94c3-4559-b98e-5cdd91f270e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.533043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f2ace-94c3-4559-b98e-5cdd91f270e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:36.534980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f2ace-94c3-4559-b98e-5cdd91f270e5 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:36.554546 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 917/1838] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:36 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:00:36.558510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8017746-7182-4814-b9cb-d951d4651678 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:36.568796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8017746-7182-4814-b9cb-d951d4651678 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:36.573908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d25d1-f189-489e-a75a-5204c3e65b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:36.574619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d25d1-f189-489e-a75a-5204c3e65b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.575010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d25d1-f189-489e-a75a-5204c3e65b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.575182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d25d1-f189-489e-a75a-5204c3e65b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.575953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d25d1-f189-489e-a75a-5204c3e65b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:36.577537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d25d1-f189-489e-a75a-5204c3e65b5a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:36.591405 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 922/1839] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:36.592853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f2ace-94c3-4559-b98e-5cdd91f270e5 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:36.603779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f2ace-94c3-4559-b98e-5cdd91f270e5 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:36.608754 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cd42a-53fa-4f9e-9178-96da172eb81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:36.610443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cd42a-53fa-4f9e-9178-96da172eb81d admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.610443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cd42a-53fa-4f9e-9178-96da172eb81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.610754 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cd42a-53fa-4f9e-9178-96da172eb81d admin admin] PATH_INFO: `/v3/projects/ddb7900e23be47869eee7ae6a0b2c844/users/8cb35404697642a991f039e65dcf32cc/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.636331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cd42a-53fa-4f9e-9178-96da172eb81d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb7900e23be47869eee7ae6a0b2c844, user_id=8cb35404697642a991f039e65dcf32cc, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:36.639065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cd42a-53fa-4f9e-9178-96da172eb81d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:36.655322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 918/1840] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:36 2022] PUT /identity/v3/projects/ddb7900e23be47869eee7ae6a0b2c844/users/8cb35404697642a991f039e65dcf32cc/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:36.659233 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d25d1-f189-489e-a75a-5204c3e65b5a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:36.669783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d25d1-f189-489e-a75a-5204c3e65b5a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:36.723075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d32280-5e7b-4d01-b554-546befdffbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:36.723741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d32280-5e7b-4d01-b554-546befdffbaf admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.724019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d32280-5e7b-4d01-b554-546befdffbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.724288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d32280-5e7b-4d01-b554-546befdffbaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.724998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d32280-5e7b-4d01-b554-546befdffbaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:36.726616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d32280-5e7b-4d01-b554-546befdffbaf admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:36.736706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 923/1841] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:36.740900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cd42a-53fa-4f9e-9178-96da172eb81d admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:36.751117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cd42a-53fa-4f9e-9178-96da172eb81d admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:36.755647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec0be5-138e-4ea0-a8f4-073966dc74c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:36.756544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0be5-138e-4ea0-a8f4-073966dc74c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.756826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0be5-138e-4ea0-a8f4-073966dc74c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.757098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0be5-138e-4ea0-a8f4-073966dc74c2 admin admin] PATH_INFO: `/v3/projects/ddb7900e23be47869eee7ae6a0b2c844/users/8cb35404697642a991f039e65dcf32cc/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.759063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec0be5-138e-4ea0-a8f4-073966dc74c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb7900e23be47869eee7ae6a0b2c844, user_id=8cb35404697642a991f039e65dcf32cc, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:36.761668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec0be5-138e-4ea0-a8f4-073966dc74c2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:36.771542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 919/1842] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:36 2022] PUT /identity/v3/projects/ddb7900e23be47869eee7ae6a0b2c844/users/8cb35404697642a991f039e65dcf32cc/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:36.782759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbbe52-0dc7-47a1-a8eb-cbbf52ebb30b None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.783053 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbbe52-0dc7-47a1-a8eb-cbbf52ebb30b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.783342 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbbe52-0dc7-47a1-a8eb-cbbf52ebb30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.789102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:36.811574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-d6cbbe52-0dc7-47a1-a8eb-cbbf52ebb30b None None] MFA Rules not processed for user `8cb35404697642a991f039e65dcf32cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:36.812758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d6cbbe52-0dc7-47a1-a8eb-cbbf52ebb30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:36.879283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 924/1843] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:36.890007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec0be5-138e-4ea0-a8f4-073966dc74c2 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:36.891750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4eec0be5-138e-4ea0-a8f4-073966dc74c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:36.953205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec0be5-138e-4ea0-a8f4-073966dc74c2 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:36.956831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ef454-6006-401d-9797-0a03545ceac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:36.960588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ef454-6006-401d-9797-0a03545ceac5 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:36.960912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ef454-6006-401d-9797-0a03545ceac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:36.961161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ef454-6006-401d-9797-0a03545ceac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:36.978824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:37.018872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ef454-6006-401d-9797-0a03545ceac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:37.020308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ef454-6006-401d-9797-0a03545ceac5 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:37.029174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 920/1844] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:46.876333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=33 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:47.025161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=19 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:47.389292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e5210-208b-4c5d-a947-d1e1688da85f None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:47.389583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e5210-208b-4c5d-a947-d1e1688da85f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:47.392345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e5210-208b-4c5d-a947-d1e1688da85f None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:47.393969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 925/1845] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 13:00:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:00:47.413893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f609f-c404-40c3-819a-7bab60a1cefb None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:47.415476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f609f-c404-40c3-819a-7bab60a1cefb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:47.415476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f609f-c404-40c3-819a-7bab60a1cefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:47.451918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1d4f609f-c404-40c3-819a-7bab60a1cefb None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:47.453660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1d4f609f-c404-40c3-819a-7bab60a1cefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:47.578611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 921/1846] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 13:00:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:47.594491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e5210-208b-4c5d-a947-d1e1688da85f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:47.594491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-aa5e5210-208b-4c5d-a947-d1e1688da85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:47.617677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e5210-208b-4c5d-a947-d1e1688da85f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:47.625160 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3201e-f5c0-4c51-a7dd-7aaf1c7d7c09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:47.625839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3201e-f5c0-4c51-a7dd-7aaf1c7d7c09 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:47.626230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3201e-f5c0-4c51-a7dd-7aaf1c7d7c09 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:47.626554 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3201e-f5c0-4c51-a7dd-7aaf1c7d7c09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:47.637527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3201e-f5c0-4c51-a7dd-7aaf1c7d7c09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:47.638959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3201e-f5c0-4c51-a7dd-7aaf1c7d7c09 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:47.651917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 926/1847] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:49.159753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7333bb1-5615-45d9-a994-fc68fb55dde4 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:49.160289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7333bb1-5615-45d9-a994-fc68fb55dde4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:49.160289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7333bb1-5615-45d9-a994-fc68fb55dde4 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:49.161710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 922/1848] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 13:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:00:49.197975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781c30-9ad0-4145-9609-4150f5db1bb9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:49.199963 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781c30-9ad0-4145-9609-4150f5db1bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:49.200427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781c30-9ad0-4145-9609-4150f5db1bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:49.235332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ea781c30-9ad0-4145-9609-4150f5db1bb9 None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:49.238187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ea781c30-9ad0-4145-9609-4150f5db1bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:49.256596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 927/1849] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 13:00:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:49.275886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7333bb1-5615-45d9-a994-fc68fb55dde4 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:49.299674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7333bb1-5615-45d9-a994-fc68fb55dde4 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:49.305721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e03c5-dd6d-4fce-8111-ec7da2e89e5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:49.306447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e03c5-dd6d-4fce-8111-ec7da2e89e5b service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:49.306725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e03c5-dd6d-4fce-8111-ec7da2e89e5b service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:49.307002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e03c5-dd6d-4fce-8111-ec7da2e89e5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:49.321979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e03c5-dd6d-4fce-8111-ec7da2e89e5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:49.324062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e03c5-dd6d-4fce-8111-ec7da2e89e5b service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:49.338589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 923/1850] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:54.630648 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea781c30-9ad0-4145-9609-4150f5db1bb9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:54.643903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea781c30-9ad0-4145-9609-4150f5db1bb9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:54.652474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585bc0d-be88-46ca-8f52-8d82e1774ff5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:54.655785 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585bc0d-be88-46ca-8f52-8d82e1774ff5 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:54.656175 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585bc0d-be88-46ca-8f52-8d82e1774ff5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:54.658936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585bc0d-be88-46ca-8f52-8d82e1774ff5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:54.674705 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585bc0d-be88-46ca-8f52-8d82e1774ff5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:54.676607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585bc0d-be88-46ca-8f52-8d82e1774ff5 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:54.712606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 928/1851] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:54.847563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e03c5-dd6d-4fce-8111-ec7da2e89e5b service nova] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:54.849752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-771e03c5-dd6d-4fce-8111-ec7da2e89e5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:55.016882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e03c5-dd6d-4fce-8111-ec7da2e89e5b service nova] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:55.028866 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:55.030661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:55.032540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:55.034039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] PATH_INFO: `/v3/users/262da4b6c9c74da691011079e34d736d` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:55.035616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=262da4b6c9c74da691011079e34d736d)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:55.038133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:55.173016 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '262da4b6c9c74da691011079e34d736d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:55.199686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '262da4b6c9c74da691011079e34d736d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:55.219468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '262da4b6c9c74da691011079e34d736d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:55.219796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '262da4b6c9c74da691011079e34d736d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:55.239282 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '262da4b6c9c74da691011079e34d736d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:55.241540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:55.251329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 924/1852] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:54 2022] DELETE /identity/v3/users/262da4b6c9c74da691011079e34d736d => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:55.256569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8cb0a-c3a6-4b59-8c08-ee2eb654ee2c None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:55.256873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8cb0a-c3a6-4b59-8c08-ee2eb654ee2c None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:55.257168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8cb0a-c3a6-4b59-8c08-ee2eb654ee2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:55.288147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e1e8cb0a-c3a6-4b59-8c08-ee2eb654ee2c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:55.290104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e1e8cb0a-c3a6-4b59-8c08-ee2eb654ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:55.368498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 929/1853] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:55.379238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:55.380834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:55.503759 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea87cb-e884-4960-a28c-e010cb7b9329 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:55.511697 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f047ba1-b374-4e28-9ce7-13b48c884f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:55.512582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f047ba1-b374-4e28-9ce7-13b48c884f17 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:55.513009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f047ba1-b374-4e28-9ce7-13b48c884f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:55.513416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f047ba1-b374-4e28-9ce7-13b48c884f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:55.527502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f047ba1-b374-4e28-9ce7-13b48c884f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:55.529027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f047ba1-b374-4e28-9ce7-13b48c884f17 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:55.545465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 925/1854] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:55.940897 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8cb0a-c3a6-4b59-8c08-ee2eb654ee2c None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:55.942122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e1e8cb0a-c3a6-4b59-8c08-ee2eb654ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:55.962550 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8cb0a-c3a6-4b59-8c08-ee2eb654ee2c None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:55.967740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:55.968839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:55.969295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:55.969724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] PATH_INFO: `/v3/projects/3a1eb621a30640b0a29acc3fa12893f4` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:55.970966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a1eb621a30640b0a29acc3fa12893f4)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:55.973001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:56.036461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a1eb621a30640b0a29acc3fa12893f4', 'request_id': 'req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.050553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a1eb621a30640b0a29acc3fa12893f4', 'request_id': 'req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.060641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a1eb621a30640b0a29acc3fa12893f4', 'request_id': 'req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.078861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] The token cache is being invalidate because project 3a1eb621a30640b0a29acc3fa12893f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:56.079107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.079762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.086347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 930/1855] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:55 2022] DELETE /identity/v3/projects/3a1eb621a30640b0a29acc3fa12893f4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:56.087157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f047ba1-b374-4e28-9ce7-13b48c884f17 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:56.114679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f047ba1-b374-4e28-9ce7-13b48c884f17 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:56.186369 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeb59e-51aa-4bca-a963-c00b626ecef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:56.187417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeb59e-51aa-4bca-a963-c00b626ecef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:56.187672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeb59e-51aa-4bca-a963-c00b626ecef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:56.188005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeb59e-51aa-4bca-a963-c00b626ecef3 admin admin] PATH_INFO: `/v3/users/65dd9f32cf98448abae4cd380ca24fae` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:56.189322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eeb59e-51aa-4bca-a963-c00b626ecef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65dd9f32cf98448abae4cd380ca24fae)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:56.194658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eeb59e-51aa-4bca-a963-c00b626ecef3 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:56.241048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-21eeb59e-51aa-4bca-a963-c00b626ecef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65dd9f32cf98448abae4cd380ca24fae'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.253856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-21eeb59e-51aa-4bca-a963-c00b626ecef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65dd9f32cf98448abae4cd380ca24fae'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.262538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-21eeb59e-51aa-4bca-a963-c00b626ecef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65dd9f32cf98448abae4cd380ca24fae'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.263292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-21eeb59e-51aa-4bca-a963-c00b626ecef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65dd9f32cf98448abae4cd380ca24fae'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.270454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-21eeb59e-51aa-4bca-a963-c00b626ecef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65dd9f32cf98448abae4cd380ca24fae'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.277402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 926/1856] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:00:56 2022] DELETE /identity/v3/users/65dd9f32cf98448abae4cd380ca24fae => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:56.581984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:56.583508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:56.603221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a0ff5-962b-43bf-90bb-ede8553a9cce admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:56.668424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:56.680619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:56.681478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:56.681760 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:56.682034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] PATH_INFO: `/v3/projects/b738714e83ff457e940b51fcced26578` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:56.683558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b738714e83ff457e940b51fcced26578)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:56.685473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:56.732195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b738714e83ff457e940b51fcced26578', 'request_id': 'req-78750395-5293-4070-aa1a-db6d4b7b57e3'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.739420 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b738714e83ff457e940b51fcced26578', 'request_id': 'req-78750395-5293-4070-aa1a-db6d4b7b57e3'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.745573 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b738714e83ff457e940b51fcced26578', 'request_id': 'req-78750395-5293-4070-aa1a-db6d4b7b57e3'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.758220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] The token cache is being invalidate because project b738714e83ff457e940b51fcced26578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:00:56.759038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.759690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-78750395-5293-4070-aa1a-db6d4b7b57e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:00:56.761566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 931/1857] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:00:56 2022] DELETE /identity/v3/projects/b738714e83ff457e940b51fcced26578 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:56.766612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048232f-d093-44b4-a700-76ba49b721e3 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:56.767002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048232f-d093-44b4-a700-76ba49b721e3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:56.768218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048232f-d093-44b4-a700-76ba49b721e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:56.786533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9048232f-d093-44b4-a700-76ba49b721e3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:56.787837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9048232f-d093-44b4-a700-76ba49b721e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:56.801520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:56.851070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 927/1858] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:00:56.858788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21a62d-6f79-4dc0-b89b-90c90dd1206a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:56.859101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21a62d-6f79-4dc0-b89b-90c90dd1206a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:56.859428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21a62d-6f79-4dc0-b89b-90c90dd1206a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:56.879891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-df21a62d-6f79-4dc0-b89b-90c90dd1206a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:56.881292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-df21a62d-6f79-4dc0-b89b-90c90dd1206a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:56.890047 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 932/1859] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:00:56.894348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048232f-d093-44b4-a700-76ba49b721e3 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:56.905501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048232f-d093-44b4-a700-76ba49b721e3 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:56.911151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc875ff3-547d-48ee-b448-4af565ad243c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:56.911846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc875ff3-547d-48ee-b448-4af565ad243c admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:56.912196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc875ff3-547d-48ee-b448-4af565ad243c admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:56.912616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc875ff3-547d-48ee-b448-4af565ad243c admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:56.913443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc875ff3-547d-48ee-b448-4af565ad243c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:00:56.913827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc875ff3-547d-48ee-b448-4af565ad243c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:56.915584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc875ff3-547d-48ee-b448-4af565ad243c admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:56.928944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 928/1860] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:00:56.935276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21a62d-6f79-4dc0-b89b-90c90dd1206a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:56.947449 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21a62d-6f79-4dc0-b89b-90c90dd1206a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:56.954304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf419f-8f1d-41d7-b44e-2f5f1e800575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:56.955060 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf419f-8f1d-41d7-b44e-2f5f1e800575 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:56.955591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf419f-8f1d-41d7-b44e-2f5f1e800575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:56.955870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf419f-8f1d-41d7-b44e-2f5f1e800575 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:56.956873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf419f-8f1d-41d7-b44e-2f5f1e800575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:56.959032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf419f-8f1d-41d7-b44e-2f5f1e800575 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:56.988991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 933/1861] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:56 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:56.993734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc875ff3-547d-48ee-b448-4af565ad243c admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:57.007232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc875ff3-547d-48ee-b448-4af565ad243c admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:57.072775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5aaf0c-1017-48c4-a43b-2ea8990545cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:57.073467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5aaf0c-1017-48c4-a43b-2ea8990545cc admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.073752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5aaf0c-1017-48c4-a43b-2ea8990545cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.074031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5aaf0c-1017-48c4-a43b-2ea8990545cc admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.074778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5aaf0c-1017-48c4-a43b-2ea8990545cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:57.076706 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5aaf0c-1017-48c4-a43b-2ea8990545cc admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:57.103591 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 929/1862] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:56 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:57.107969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf419f-8f1d-41d7-b44e-2f5f1e800575 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:57.119939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf419f-8f1d-41d7-b44e-2f5f1e800575 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:57.124876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cde44-25c5-4f6f-9610-ef18bd0d5086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:57.125535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cde44-25c5-4f6f-9610-ef18bd0d5086 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.125813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cde44-25c5-4f6f-9610-ef18bd0d5086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.126209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cde44-25c5-4f6f-9610-ef18bd0d5086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.126966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394cde44-25c5-4f6f-9610-ef18bd0d5086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:57.129073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394cde44-25c5-4f6f-9610-ef18bd0d5086 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:57.146089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 934/1863] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:00:57.147067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5aaf0c-1017-48c4-a43b-2ea8990545cc admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:57.160475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5aaf0c-1017-48c4-a43b-2ea8990545cc admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:57.167116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d724a-0d73-4b9b-ab0f-92bc6cef8e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:57.167795 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d724a-0d73-4b9b-ab0f-92bc6cef8e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.168082 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d724a-0d73-4b9b-ab0f-92bc6cef8e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.168352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d724a-0d73-4b9b-ab0f-92bc6cef8e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.169086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d724a-0d73-4b9b-ab0f-92bc6cef8e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:57.170706 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d724a-0d73-4b9b-ab0f-92bc6cef8e71 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:57.182326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 930/1864] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:57.185588 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cde44-25c5-4f6f-9610-ef18bd0d5086 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:57.197584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cde44-25c5-4f6f-9610-ef18bd0d5086 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:57.203993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e38beb-367d-44ef-be5c-64db5e8659d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:57.205515 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e38beb-367d-44ef-be5c-64db5e8659d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.205800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e38beb-367d-44ef-be5c-64db5e8659d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.206077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e38beb-367d-44ef-be5c-64db5e8659d5 admin admin] PATH_INFO: `/v3/projects/46f888e6147d4bb88824155a2d19deee/users/7708c98e12364a39801512c0e72d3d67/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.234060 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e38beb-367d-44ef-be5c-64db5e8659d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f888e6147d4bb88824155a2d19deee, user_id=7708c98e12364a39801512c0e72d3d67, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:57.238269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e38beb-367d-44ef-be5c-64db5e8659d5 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:57.256049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 935/1865] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:57 2022] PUT /identity/v3/projects/46f888e6147d4bb88824155a2d19deee/users/7708c98e12364a39801512c0e72d3d67/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:00:57.268824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190810c-ca2b-4d1b-9636-52cba4b6a9bb None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.269241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190810c-ca2b-4d1b-9636-52cba4b6a9bb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.269676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190810c-ca2b-4d1b-9636-52cba4b6a9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.299885 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7190810c-ca2b-4d1b-9636-52cba4b6a9bb None None] MFA Rules not processed for user `7708c98e12364a39801512c0e72d3d67`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:57.301560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7190810c-ca2b-4d1b-9636-52cba4b6a9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:57.381466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 931/1866] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:57.386228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e38beb-367d-44ef-be5c-64db5e8659d5 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:57.451395 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e38beb-367d-44ef-be5c-64db5e8659d5 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:57.456071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb73a4-85ad-4380-b610-a678dc4259fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:57.456729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb73a4-85ad-4380-b610-a678dc4259fa admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.457010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb73a4-85ad-4380-b610-a678dc4259fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.457284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb73a4-85ad-4380-b610-a678dc4259fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.457967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb73a4-85ad-4380-b610-a678dc4259fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:57.459938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb73a4-85ad-4380-b610-a678dc4259fa admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:57.484707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 936/1867] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:00:57 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:00:57.489482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190810c-ca2b-4d1b-9636-52cba4b6a9bb None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:57.501823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190810c-ca2b-4d1b-9636-52cba4b6a9bb None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:57.513708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:57.558645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e11a83-c395-4b3a-9051-2a9bc2e174db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:57.559319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e11a83-c395-4b3a-9051-2a9bc2e174db admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.559608 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e11a83-c395-4b3a-9051-2a9bc2e174db admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.559890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e11a83-c395-4b3a-9051-2a9bc2e174db admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.560593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e11a83-c395-4b3a-9051-2a9bc2e174db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:57.562518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e11a83-c395-4b3a-9051-2a9bc2e174db admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:57.581405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 932/1868] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:00:57 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:00:57.585363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb73a4-85ad-4380-b610-a678dc4259fa admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:57.591357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:00:57.597732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb73a4-85ad-4380-b610-a678dc4259fa admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:57.603773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd561e-ae59-4641-aa03-0bdf748c644a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:57.604428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd561e-ae59-4641-aa03-0bdf748c644a admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.604713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd561e-ae59-4641-aa03-0bdf748c644a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.604988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd561e-ae59-4641-aa03-0bdf748c644a admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.605703 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd561e-ae59-4641-aa03-0bdf748c644a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:57.607335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd561e-ae59-4641-aa03-0bdf748c644a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:57.618514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 937/1869] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:00:57.622954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e11a83-c395-4b3a-9051-2a9bc2e174db admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:57.632928 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e11a83-c395-4b3a-9051-2a9bc2e174db admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:57.637944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e01bb-12c2-4ea8-82e2-187e84e54cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:57.638881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e01bb-12c2-4ea8-82e2-187e84e54cad admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.639168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e01bb-12c2-4ea8-82e2-187e84e54cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.639447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e01bb-12c2-4ea8-82e2-187e84e54cad admin admin] PATH_INFO: `/v3/projects/019ec3bf0eb74547b20eb762b0b0efe7/users/b9b76c76c3c1406390f21108bc40fb16/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.657147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e01bb-12c2-4ea8-82e2-187e84e54cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=019ec3bf0eb74547b20eb762b0b0efe7, user_id=b9b76c76c3c1406390f21108bc40fb16, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:57.660151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e01bb-12c2-4ea8-82e2-187e84e54cad admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:57.684448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 933/1870] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:57 2022] PUT /identity/v3/projects/019ec3bf0eb74547b20eb762b0b0efe7/users/b9b76c76c3c1406390f21108bc40fb16/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:57.688604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd561e-ae59-4641-aa03-0bdf748c644a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:57.699514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd561e-ae59-4641-aa03-0bdf748c644a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:57.757830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9bca2-6b27-4652-b0eb-d2e34bd492dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:57.758549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9bca2-6b27-4652-b0eb-d2e34bd492dc admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.758837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9bca2-6b27-4652-b0eb-d2e34bd492dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.759115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9bca2-6b27-4652-b0eb-d2e34bd492dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.759838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9bca2-6b27-4652-b0eb-d2e34bd492dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:57.761472 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9bca2-6b27-4652-b0eb-d2e34bd492dc admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:57.771090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 938/1871] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:00:57.775736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e01bb-12c2-4ea8-82e2-187e84e54cad admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:57.786019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e01bb-12c2-4ea8-82e2-187e84e54cad admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:57.794927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4d274-2755-49ff-8640-f65cbc77613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:57.795825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d274-2755-49ff-8640-f65cbc77613e admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.796110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d274-2755-49ff-8640-f65cbc77613e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.796389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d274-2755-49ff-8640-f65cbc77613e admin admin] PATH_INFO: `/v3/projects/019ec3bf0eb74547b20eb762b0b0efe7/users/b9b76c76c3c1406390f21108bc40fb16/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.798431 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4d274-2755-49ff-8640-f65cbc77613e admin admin] RBAC: Authorizing `identity:create_grant(project_id=019ec3bf0eb74547b20eb762b0b0efe7, user_id=b9b76c76c3c1406390f21108bc40fb16, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:57.801074 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4d274-2755-49ff-8640-f65cbc77613e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:57.813564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 934/1872] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:00:57 2022] PUT /identity/v3/projects/019ec3bf0eb74547b20eb762b0b0efe7/users/b9b76c76c3c1406390f21108bc40fb16/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:00:57.825079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cdcbe-cbb7-4a0f-977e-5c44c0cb75e0 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.825370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cdcbe-cbb7-4a0f-977e-5c44c0cb75e0 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.825647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cdcbe-cbb7-4a0f-977e-5c44c0cb75e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:57.852197 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-077cdcbe-cbb7-4a0f-977e-5c44c0cb75e0 None None] MFA Rules not processed for user `b9b76c76c3c1406390f21108bc40fb16`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:00:57.853394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-077cdcbe-cbb7-4a0f-977e-5c44c0cb75e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:57.919795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 939/1873] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:00:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:00:57.930818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4d274-2755-49ff-8640-f65cbc77613e admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:57.932517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d2b4d274-2755-49ff-8640-f65cbc77613e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:00:57.989947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4d274-2755-49ff-8640-f65cbc77613e admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:57.993669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a91008-eb85-4781-aae8-585863524341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:57.994337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91008-eb85-4781-aae8-585863524341 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:57.994630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91008-eb85-4781-aae8-585863524341 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:57.994913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91008-eb85-4781-aae8-585863524341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:58.004098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a91008-eb85-4781-aae8-585863524341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:58.005520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a91008-eb85-4781-aae8-585863524341 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:58.015807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 935/1874] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:00:58.096704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cdcbe-cbb7-4a0f-977e-5c44c0cb75e0 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:00:58.106406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cdcbe-cbb7-4a0f-977e-5c44c0cb75e0 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:00:58.110095 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29edd03-6ca9-4e8c-a014-8aa7e26a49a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:00:58.110796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29edd03-6ca9-4e8c-a014-8aa7e26a49a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:00:58.111079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29edd03-6ca9-4e8c-a014-8aa7e26a49a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:00:58.111420 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29edd03-6ca9-4e8c-a014-8aa7e26a49a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:00:58.182551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29edd03-6ca9-4e8c-a014-8aa7e26a49a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:00:58.183924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29edd03-6ca9-4e8c-a014-8aa7e26a49a8 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:00:58.193335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 940/1875] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:00:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:08.011632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=63,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:08.188843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=95 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:18.760486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a91008-eb85-4781-aae8-585863524341 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:18.837127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a91008-eb85-4781-aae8-585863524341 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:18.842371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:18.843485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:18.843845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:18.844195 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] PATH_INFO: `/v3/users/7708c98e12364a39801512c0e72d3d67` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:18.845757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7708c98e12364a39801512c0e72d3d67)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:18.847951 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:18.922081 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7708c98e12364a39801512c0e72d3d67'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:18.936767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7708c98e12364a39801512c0e72d3d67'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:18.948098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7708c98e12364a39801512c0e72d3d67'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:18.948774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7708c98e12364a39801512c0e72d3d67'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:18.959438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7708c98e12364a39801512c0e72d3d67'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:18.967283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 936/1876] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:18 2022] DELETE /identity/v3/users/7708c98e12364a39801512c0e72d3d67 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:18.971550 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c629cc-2fe6-4bff-bcb9-de686bf1fbb7 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:18.972091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c629cc-2fe6-4bff-bcb9-de686bf1fbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:18.972091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c629cc-2fe6-4bff-bcb9-de686bf1fbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:18.990092 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f6c629cc-2fe6-4bff-bcb9-de686bf1fbb7 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:18.991464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f6c629cc-2fe6-4bff-bcb9-de686bf1fbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:19.063936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 941/1877] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:19.073845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:19.075289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:19.130626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bfbe8-a7f9-455d-bb12-8874365ae76c admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:19.135258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acab02-9fed-41f9-9b76-650ff474cb4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:19.135899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acab02-9fed-41f9-9b76-650ff474cb4a service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:19.136183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acab02-9fed-41f9-9b76-650ff474cb4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:19.136458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acab02-9fed-41f9-9b76-650ff474cb4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:19.145419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acab02-9fed-41f9-9b76-650ff474cb4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:19.146899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acab02-9fed-41f9-9b76-650ff474cb4a service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:19.157740 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 937/1878] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:01:19.447920 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c629cc-2fe6-4bff-bcb9-de686bf1fbb7 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:19.449453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f6c629cc-2fe6-4bff-bcb9-de686bf1fbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:19.479112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c629cc-2fe6-4bff-bcb9-de686bf1fbb7 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:19.484915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:19.485793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:19.486076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:19.486397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] PATH_INFO: `/v3/projects/46f888e6147d4bb88824155a2d19deee` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:19.487607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f888e6147d4bb88824155a2d19deee)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:19.489545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:19.544723 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f888e6147d4bb88824155a2d19deee', 'request_id': 'req-a18347ee-68a5-4e75-b453-91fa2a48cf3f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:19.552387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f888e6147d4bb88824155a2d19deee', 'request_id': 'req-a18347ee-68a5-4e75-b453-91fa2a48cf3f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:19.558761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f888e6147d4bb88824155a2d19deee', 'request_id': 'req-a18347ee-68a5-4e75-b453-91fa2a48cf3f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:19.569740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] The token cache is being invalidate because project 46f888e6147d4bb88824155a2d19deee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:19.570059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:19.570724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:19.572275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 942/1879] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:19 2022] DELETE /identity/v3/projects/46f888e6147d4bb88824155a2d19deee => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:19.576254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acab02-9fed-41f9-9b76-650ff474cb4a service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:19.585848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acab02-9fed-41f9-9b76-650ff474cb4a service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:19.604621 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:19.648739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294df0c7-4282-4c31-a682-b8df09b2ee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:19.648739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294df0c7-4282-4c31-a682-b8df09b2ee99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:19.648739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294df0c7-4282-4c31-a682-b8df09b2ee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:19.648739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294df0c7-4282-4c31-a682-b8df09b2ee99 admin admin] PATH_INFO: `/v3/users/b9b76c76c3c1406390f21108bc40fb16` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:19.649088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294df0c7-4282-4c31-a682-b8df09b2ee99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9b76c76c3c1406390f21108bc40fb16)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:19.649812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294df0c7-4282-4c31-a682-b8df09b2ee99 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:19.696521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-294df0c7-4282-4c31-a682-b8df09b2ee99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9b76c76c3c1406390f21108bc40fb16'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:19.708048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-294df0c7-4282-4c31-a682-b8df09b2ee99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9b76c76c3c1406390f21108bc40fb16'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:19.716054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-294df0c7-4282-4c31-a682-b8df09b2ee99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9b76c76c3c1406390f21108bc40fb16'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:19.716701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-294df0c7-4282-4c31-a682-b8df09b2ee99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9b76c76c3c1406390f21108bc40fb16'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:19.723404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-294df0c7-4282-4c31-a682-b8df09b2ee99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9b76c76c3c1406390f21108bc40fb16'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:19.729822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 938/1880] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:19 2022] DELETE /identity/v3/users/b9b76c76c3c1406390f21108bc40fb16 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:20.055839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:20.057973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:20.072890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18347ee-68a5-4e75-b453-91fa2a48cf3f admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:20.140051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:20.140925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.141206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.141480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] PATH_INFO: `/v3/projects/019ec3bf0eb74547b20eb762b0b0efe7` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.142754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=019ec3bf0eb74547b20eb762b0b0efe7)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:20.144691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:20.181417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:20.205140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '019ec3bf0eb74547b20eb762b0b0efe7', 'request_id': 'req-a10fd3ea-cf05-496b-827e-e848e01559b6'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:20.215820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '019ec3bf0eb74547b20eb762b0b0efe7', 'request_id': 'req-a10fd3ea-cf05-496b-827e-e848e01559b6'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:20.221633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '019ec3bf0eb74547b20eb762b0b0efe7', 'request_id': 'req-a10fd3ea-cf05-496b-827e-e848e01559b6'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:20.233212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] The token cache is being invalidate because project 019ec3bf0eb74547b20eb762b0b0efe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:20.233506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:20.234185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a10fd3ea-cf05-496b-827e-e848e01559b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:20.235993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 943/1881] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:20 2022] DELETE /identity/v3/projects/019ec3bf0eb74547b20eb762b0b0efe7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:20.240676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6a429-35d6-4828-ab5b-11f09736dff8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.241145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6a429-35d6-4828-ab5b-11f09736dff8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.241381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6a429-35d6-4828-ab5b-11f09736dff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.260674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-22d6a429-35d6-4828-ab5b-11f09736dff8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:20.261894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-22d6a429-35d6-4828-ab5b-11f09736dff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:20.322069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 939/1882] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:01:20.329629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683577a0-2088-4107-825c-a68611d27209 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.329865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683577a0-2088-4107-825c-a68611d27209 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.330150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683577a0-2088-4107-825c-a68611d27209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.348016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-683577a0-2088-4107-825c-a68611d27209 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:20.349210 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-683577a0-2088-4107-825c-a68611d27209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:20.357569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 944/1883] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:20.361535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6a429-35d6-4828-ab5b-11f09736dff8 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:20.371872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6a429-35d6-4828-ab5b-11f09736dff8 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:20.376395 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a96566-265a-431d-8249-792b178bd51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:20.377047 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a96566-265a-431d-8249-792b178bd51e admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.377331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a96566-265a-431d-8249-792b178bd51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.377604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a96566-265a-431d-8249-792b178bd51e admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.378384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a96566-265a-431d-8249-792b178bd51e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:01:20.378691 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a96566-265a-431d-8249-792b178bd51e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:20.380285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a96566-265a-431d-8249-792b178bd51e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:20.395016 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 940/1884] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:01:20.396305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683577a0-2088-4107-825c-a68611d27209 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:20.405827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683577a0-2088-4107-825c-a68611d27209 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:20.410542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152c924-f02c-4328-9b7f-2a94438065c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:20.411289 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152c924-f02c-4328-9b7f-2a94438065c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.411477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152c924-f02c-4328-9b7f-2a94438065c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.411751 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152c924-f02c-4328-9b7f-2a94438065c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.412445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152c924-f02c-4328-9b7f-2a94438065c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:20.415087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152c924-f02c-4328-9b7f-2a94438065c3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:20.447750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 945/1885] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:20 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:20.448928 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a96566-265a-431d-8249-792b178bd51e admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:20.458780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a96566-265a-431d-8249-792b178bd51e admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:20.489555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:20.521815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d278d4-f4ad-4b5c-9164-7826870239a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:20.522525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d278d4-f4ad-4b5c-9164-7826870239a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.522833 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d278d4-f4ad-4b5c-9164-7826870239a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.523140 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d278d4-f4ad-4b5c-9164-7826870239a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.523826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d278d4-f4ad-4b5c-9164-7826870239a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:20.525733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d278d4-f4ad-4b5c-9164-7826870239a2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:20.545713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 941/1886] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:20 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:20.550016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152c924-f02c-4328-9b7f-2a94438065c3 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:20.561133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152c924-f02c-4328-9b7f-2a94438065c3 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:20.566066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f3ca-5be0-429b-8631-93d6c76ff22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:20.566808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f3ca-5be0-429b-8631-93d6c76ff22f admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.567093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f3ca-5be0-429b-8631-93d6c76ff22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.567373 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f3ca-5be0-429b-8631-93d6c76ff22f admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.568104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402f3ca-5be0-429b-8631-93d6c76ff22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:20.569722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402f3ca-5be0-429b-8631-93d6c76ff22f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:20.580628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 946/1887] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:20.584546 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d278d4-f4ad-4b5c-9164-7826870239a2 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:20.595207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d278d4-f4ad-4b5c-9164-7826870239a2 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:20.600142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a3d53-36e4-4a0d-8a0a-4d6bc3a0d5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:20.600872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a3d53-36e4-4a0d-8a0a-4d6bc3a0d5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.601652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a3d53-36e4-4a0d-8a0a-4d6bc3a0d5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.601978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a3d53-36e4-4a0d-8a0a-4d6bc3a0d5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.602726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a3d53-36e4-4a0d-8a0a-4d6bc3a0d5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:20.604405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a3d53-36e4-4a0d-8a0a-4d6bc3a0d5c4 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:20.617544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 942/1888] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:20.622059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f3ca-5be0-429b-8631-93d6c76ff22f admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:20.631991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f3ca-5be0-429b-8631-93d6c76ff22f admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:20.636555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7d0ba-899b-4173-8c7f-8c6b08ebd087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:20.637428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7d0ba-899b-4173-8c7f-8c6b08ebd087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.637714 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7d0ba-899b-4173-8c7f-8c6b08ebd087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.637988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7d0ba-899b-4173-8c7f-8c6b08ebd087 admin admin] PATH_INFO: `/v3/projects/6d7be97775bc41e5a77f6e18f4e3a1aa/users/787fe096f8954f23b43fe9e5107b1edb/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.652727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7d0ba-899b-4173-8c7f-8c6b08ebd087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d7be97775bc41e5a77f6e18f4e3a1aa, user_id=787fe096f8954f23b43fe9e5107b1edb, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:20.655544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7d0ba-899b-4173-8c7f-8c6b08ebd087 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:20.673135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 947/1889] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:20 2022] PUT /identity/v3/projects/6d7be97775bc41e5a77f6e18f4e3a1aa/users/787fe096f8954f23b43fe9e5107b1edb/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:20.684422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232992e-1e77-41a4-945e-c8180960a76c None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.684718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232992e-1e77-41a4-945e-c8180960a76c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.685034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232992e-1e77-41a4-945e-c8180960a76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.711686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5232992e-1e77-41a4-945e-c8180960a76c None None] MFA Rules not processed for user `787fe096f8954f23b43fe9e5107b1edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:20.712876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5232992e-1e77-41a4-945e-c8180960a76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:20.776569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 943/1890] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:20.781280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7d0ba-899b-4173-8c7f-8c6b08ebd087 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:20.843173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7d0ba-899b-4173-8c7f-8c6b08ebd087 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:20.847809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec0a8f-1e90-447b-9b53-ab5df3b37368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:20.848471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec0a8f-1e90-447b-9b53-ab5df3b37368 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.848758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec0a8f-1e90-447b-9b53-ab5df3b37368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.849043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec0a8f-1e90-447b-9b53-ab5df3b37368 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.849729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec0a8f-1e90-447b-9b53-ab5df3b37368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:20.851685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec0a8f-1e90-447b-9b53-ab5df3b37368 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:20.876660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 948/1891] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:20 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:20.880704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232992e-1e77-41a4-945e-c8180960a76c None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:20.891796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232992e-1e77-41a4-945e-c8180960a76c None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:20.946542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b4c74-ceb7-40a7-aedb-190474f030d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:20.947208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b4c74-ceb7-40a7-aedb-190474f030d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.947497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b4c74-ceb7-40a7-aedb-190474f030d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.947777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b4c74-ceb7-40a7-aedb-190474f030d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.948502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b4c74-ceb7-40a7-aedb-190474f030d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:20.950482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b4c74-ceb7-40a7-aedb-190474f030d3 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:20.969954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 944/1892] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:20 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:20.973864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec0a8f-1e90-447b-9b53-ab5df3b37368 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:20.984035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec0a8f-1e90-447b-9b53-ab5df3b37368 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:20.989998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc7a1d-56d7-45ea-8f7d-2b0dbe11cae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:20.990702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc7a1d-56d7-45ea-8f7d-2b0dbe11cae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:20.990982 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc7a1d-56d7-45ea-8f7d-2b0dbe11cae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:20.991285 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc7a1d-56d7-45ea-8f7d-2b0dbe11cae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:20.992544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc7a1d-56d7-45ea-8f7d-2b0dbe11cae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:20.994144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc7a1d-56d7-45ea-8f7d-2b0dbe11cae7 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:21.005444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 949/1893] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:21.010043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b4c74-ceb7-40a7-aedb-190474f030d3 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:21.020104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b4c74-ceb7-40a7-aedb-190474f030d3 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:21.024703 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ee0b4-27fa-46d8-bd63-fca530def337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:21.025582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee0b4-27fa-46d8-bd63-fca530def337 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:21.025881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee0b4-27fa-46d8-bd63-fca530def337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:21.026134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee0b4-27fa-46d8-bd63-fca530def337 admin admin] PATH_INFO: `/v3/projects/f5b527f1029545fcb38ad31ecbb39533/users/a84ef4ffdeb0490eaa73b77642792ac1/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:21.041097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ee0b4-27fa-46d8-bd63-fca530def337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b527f1029545fcb38ad31ecbb39533, user_id=a84ef4ffdeb0490eaa73b77642792ac1, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:21.043768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ee0b4-27fa-46d8-bd63-fca530def337 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:21.056442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 945/1894] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:21 2022] PUT /identity/v3/projects/f5b527f1029545fcb38ad31ecbb39533/users/a84ef4ffdeb0490eaa73b77642792ac1/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:21.060290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc7a1d-56d7-45ea-8f7d-2b0dbe11cae7 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:21.070964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc7a1d-56d7-45ea-8f7d-2b0dbe11cae7 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:21.090894 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:21.129341 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a65f28-b6e7-43ba-bb6a-8785aff6f5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:21.129953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a65f28-b6e7-43ba-bb6a-8785aff6f5fc admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:21.130266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a65f28-b6e7-43ba-bb6a-8785aff6f5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:21.130570 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a65f28-b6e7-43ba-bb6a-8785aff6f5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:21.131290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a65f28-b6e7-43ba-bb6a-8785aff6f5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:21.133396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a65f28-b6e7-43ba-bb6a-8785aff6f5fc admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:21.143697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 950/1895] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:21.148005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ee0b4-27fa-46d8-bd63-fca530def337 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:21.154176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:21.161341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ee0b4-27fa-46d8-bd63-fca530def337 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:21.166429 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf77534-e161-4376-8609-9396fb28580b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:21.167324 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77534-e161-4376-8609-9396fb28580b admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:21.167610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77534-e161-4376-8609-9396fb28580b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:21.167892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77534-e161-4376-8609-9396fb28580b admin admin] PATH_INFO: `/v3/projects/f5b527f1029545fcb38ad31ecbb39533/users/a84ef4ffdeb0490eaa73b77642792ac1/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:21.169862 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf77534-e161-4376-8609-9396fb28580b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b527f1029545fcb38ad31ecbb39533, user_id=a84ef4ffdeb0490eaa73b77642792ac1, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:21.172605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf77534-e161-4376-8609-9396fb28580b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:21.183241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 946/1896] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:21 2022] PUT /identity/v3/projects/f5b527f1029545fcb38ad31ecbb39533/users/a84ef4ffdeb0490eaa73b77642792ac1/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:21.195487 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fbf90-6020-454e-a84b-bb50a1d7e032 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:21.195786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fbf90-6020-454e-a84b-bb50a1d7e032 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:21.196063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fbf90-6020-454e-a84b-bb50a1d7e032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:21.224510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-223fbf90-6020-454e-a84b-bb50a1d7e032 None None] MFA Rules not processed for user `a84ef4ffdeb0490eaa73b77642792ac1`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:21.225852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-223fbf90-6020-454e-a84b-bb50a1d7e032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:21.301633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 951/1897] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:21.312797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf77534-e161-4376-8609-9396fb28580b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:21.315673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5cf77534-e161-4376-8609-9396fb28580b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:21.382448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf77534-e161-4376-8609-9396fb28580b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:21.386074 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4867f-be17-4f4e-a9e7-e2c772d7a6e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:21.386761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4867f-be17-4f4e-a9e7-e2c772d7a6e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:21.387045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4867f-be17-4f4e-a9e7-e2c772d7a6e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:21.387322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4867f-be17-4f4e-a9e7-e2c772d7a6e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:21.397205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4867f-be17-4f4e-a9e7-e2c772d7a6e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:21.399296 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4867f-be17-4f4e-a9e7-e2c772d7a6e4 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:21.409773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 947/1898] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:22.007366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fbf90-6020-454e-a84b-bb50a1d7e032 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:22.070510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fbf90-6020-454e-a84b-bb50a1d7e032 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:22.075030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:22.075892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:22.076171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:22.076445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] PATH_INFO: `/v3/users/787fe096f8954f23b43fe9e5107b1edb` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:22.077624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] RBAC: Authorizing `identity:delete_user(user_id=787fe096f8954f23b43fe9e5107b1edb)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:22.079671 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:22.129336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '787fe096f8954f23b43fe9e5107b1edb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:22.131666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:22.138702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '787fe096f8954f23b43fe9e5107b1edb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:22.139406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '787fe096f8954f23b43fe9e5107b1edb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:22.140213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '787fe096f8954f23b43fe9e5107b1edb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:22.152450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '787fe096f8954f23b43fe9e5107b1edb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:22.165057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 952/1899] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:22 2022] DELETE /identity/v3/users/787fe096f8954f23b43fe9e5107b1edb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:22.169886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9a1f-0342-4208-97e1-79d8c944ce01 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:22.170176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9a1f-0342-4208-97e1-79d8c944ce01 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:22.170485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9a1f-0342-4208-97e1-79d8c944ce01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:22.187607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-698b9a1f-0342-4208-97e1-79d8c944ce01 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:22.188782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-698b9a1f-0342-4208-97e1-79d8c944ce01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:22.259143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 948/1900] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:22.268968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:22.270442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:22.328380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fa372-48f9-486b-8f38-a44941bd784f admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:22.332098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb1bd9-8731-4f9a-a087-cc58881fdda0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:22.332781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb1bd9-8731-4f9a-a087-cc58881fdda0 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:22.333072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb1bd9-8731-4f9a-a087-cc58881fdda0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:22.333352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb1bd9-8731-4f9a-a087-cc58881fdda0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:22.343319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb1bd9-8731-4f9a-a087-cc58881fdda0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:22.344771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb1bd9-8731-4f9a-a087-cc58881fdda0 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:22.355261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 953/1901] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:22.822060 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b9a1f-0342-4208-97e1-79d8c944ce01 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:22.823518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-698b9a1f-0342-4208-97e1-79d8c944ce01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:22.836274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b9a1f-0342-4208-97e1-79d8c944ce01 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:22.842058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:22.842992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:22.843279 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:22.843575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] PATH_INFO: `/v3/projects/6d7be97775bc41e5a77f6e18f4e3a1aa` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:22.844738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d7be97775bc41e5a77f6e18f4e3a1aa)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:22.846663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:22.893175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d7be97775bc41e5a77f6e18f4e3a1aa', 'request_id': 'req-dc94a7a5-0c05-49e2-8268-ed42541fb343'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:22.900902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d7be97775bc41e5a77f6e18f4e3a1aa', 'request_id': 'req-dc94a7a5-0c05-49e2-8268-ed42541fb343'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:22.906496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d7be97775bc41e5a77f6e18f4e3a1aa', 'request_id': 'req-dc94a7a5-0c05-49e2-8268-ed42541fb343'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:22.917060 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] The token cache is being invalidate because project 6d7be97775bc41e5a77f6e18f4e3a1aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:22.917346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:22.917975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:22.919540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 949/1902] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:22 2022] DELETE /identity/v3/projects/6d7be97775bc41e5a77f6e18f4e3a1aa => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:22.923566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb1bd9-8731-4f9a-a087-cc58881fdda0 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:22.933896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb1bd9-8731-4f9a-a087-cc58881fdda0 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:22.988576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b66784-a828-454c-a6c0-77015dad4c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:22.989457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b66784-a828-454c-a6c0-77015dad4c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:22.989817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b66784-a828-454c-a6c0-77015dad4c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:22.990098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b66784-a828-454c-a6c0-77015dad4c97 admin admin] PATH_INFO: `/v3/users/a84ef4ffdeb0490eaa73b77642792ac1` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:22.991422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b66784-a828-454c-a6c0-77015dad4c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a84ef4ffdeb0490eaa73b77642792ac1)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:22.993803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b66784-a828-454c-a6c0-77015dad4c97 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:23.038698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-60b66784-a828-454c-a6c0-77015dad4c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a84ef4ffdeb0490eaa73b77642792ac1'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:23.045960 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-60b66784-a828-454c-a6c0-77015dad4c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a84ef4ffdeb0490eaa73b77642792ac1'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:23.046578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-60b66784-a828-454c-a6c0-77015dad4c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a84ef4ffdeb0490eaa73b77642792ac1'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:23.047185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-60b66784-a828-454c-a6c0-77015dad4c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a84ef4ffdeb0490eaa73b77642792ac1'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:23.058522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:23.059716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-60b66784-a828-454c-a6c0-77015dad4c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a84ef4ffdeb0490eaa73b77642792ac1'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:23.074076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 954/1903] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:22 2022] DELETE /identity/v3/users/a84ef4ffdeb0490eaa73b77642792ac1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:23.338426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:23.339856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:23.353278 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94a7a5-0c05-49e2-8268-ed42541fb343 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:23.365934 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:23.413353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:23.414793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:23.415076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:23.415350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] PATH_INFO: `/v3/projects/f5b527f1029545fcb38ad31ecbb39533` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:23.416548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b527f1029545fcb38ad31ecbb39533)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:23.418561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:23.464517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b527f1029545fcb38ad31ecbb39533', 'request_id': 'req-9a85be32-1bb4-4796-bb8d-301487af6d0d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:23.470838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b527f1029545fcb38ad31ecbb39533', 'request_id': 'req-9a85be32-1bb4-4796-bb8d-301487af6d0d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:23.476226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b527f1029545fcb38ad31ecbb39533', 'request_id': 'req-9a85be32-1bb4-4796-bb8d-301487af6d0d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:23.486944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] The token cache is being invalidate because project f5b527f1029545fcb38ad31ecbb39533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:23.487233 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:23.487867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9a85be32-1bb4-4796-bb8d-301487af6d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:23.489386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 950/1904] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:23 2022] DELETE /identity/v3/projects/f5b527f1029545fcb38ad31ecbb39533 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:23.495316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728ce67-417a-42cc-92b7-2370b350a322 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:23.495611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728ce67-417a-42cc-92b7-2370b350a322 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:23.495919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728ce67-417a-42cc-92b7-2370b350a322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:23.514634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-2728ce67-417a-42cc-92b7-2370b350a322 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:23.515834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2728ce67-417a-42cc-92b7-2370b350a322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:23.579872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 955/1905] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:23.588266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da295c20-c4a5-430e-b2ba-0e26eb6ed50d None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:23.588558 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da295c20-c4a5-430e-b2ba-0e26eb6ed50d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:23.588914 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da295c20-c4a5-430e-b2ba-0e26eb6ed50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:23.612009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-da295c20-c4a5-430e-b2ba-0e26eb6ed50d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:23.615571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-da295c20-c4a5-430e-b2ba-0e26eb6ed50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:23.626622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 951/1906] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:23.630843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728ce67-417a-42cc-92b7-2370b350a322 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:23.644948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728ce67-417a-42cc-92b7-2370b350a322 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:23.651715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c670f3-a401-4201-87dd-b17fc820c55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:23.652394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c670f3-a401-4201-87dd-b17fc820c55d admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:23.652681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c670f3-a401-4201-87dd-b17fc820c55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:23.653006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c670f3-a401-4201-87dd-b17fc820c55d admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:23.653702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c670f3-a401-4201-87dd-b17fc820c55d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:01:23.654014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c670f3-a401-4201-87dd-b17fc820c55d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:23.655711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c670f3-a401-4201-87dd-b17fc820c55d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:23.668846 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 956/1907] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:01:23.673265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da295c20-c4a5-430e-b2ba-0e26eb6ed50d None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:23.684976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da295c20-c4a5-430e-b2ba-0e26eb6ed50d None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:23.689792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5404035-0aa3-4d9f-88e2-4587dee265ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:23.690497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5404035-0aa3-4d9f-88e2-4587dee265ed admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:23.690782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5404035-0aa3-4d9f-88e2-4587dee265ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:23.691061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5404035-0aa3-4d9f-88e2-4587dee265ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:23.691755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5404035-0aa3-4d9f-88e2-4587dee265ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:23.693661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5404035-0aa3-4d9f-88e2-4587dee265ed admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:23.732305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 952/1908] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:23 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:23.736299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c670f3-a401-4201-87dd-b17fc820c55d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:23.747080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c670f3-a401-4201-87dd-b17fc820c55d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:23.821214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66440c68-8d75-42cd-96fa-cba6bbe99d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:23.821903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66440c68-8d75-42cd-96fa-cba6bbe99d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:23.822194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66440c68-8d75-42cd-96fa-cba6bbe99d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:23.822544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66440c68-8d75-42cd-96fa-cba6bbe99d48 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:23.823381 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66440c68-8d75-42cd-96fa-cba6bbe99d48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:23.825330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66440c68-8d75-42cd-96fa-cba6bbe99d48 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:23.852110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 957/1909] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:23 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:23.856616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5404035-0aa3-4d9f-88e2-4587dee265ed admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:23.869239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5404035-0aa3-4d9f-88e2-4587dee265ed admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:23.876709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90059da8-325c-4dbe-8333-e622b39e94f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:23.877511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90059da8-325c-4dbe-8333-e622b39e94f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:23.877822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90059da8-325c-4dbe-8333-e622b39e94f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:23.878167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90059da8-325c-4dbe-8333-e622b39e94f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:23.879036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90059da8-325c-4dbe-8333-e622b39e94f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:23.880721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90059da8-325c-4dbe-8333-e622b39e94f3 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:23.892669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 953/1910] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:23.896794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66440c68-8d75-42cd-96fa-cba6bbe99d48 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:23.907680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66440c68-8d75-42cd-96fa-cba6bbe99d48 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:23.913177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88923e-ad23-46a5-ae4e-95fa14db7d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:23.914160 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88923e-ad23-46a5-ae4e-95fa14db7d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:23.914313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88923e-ad23-46a5-ae4e-95fa14db7d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:23.914682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88923e-ad23-46a5-ae4e-95fa14db7d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:23.915294 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88923e-ad23-46a5-ae4e-95fa14db7d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:23.917727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88923e-ad23-46a5-ae4e-95fa14db7d45 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:23.933066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 958/1911] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:23.934079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90059da8-325c-4dbe-8333-e622b39e94f3 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:23.947566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90059da8-325c-4dbe-8333-e622b39e94f3 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:23.952575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64e977-a0c7-41e0-8b39-457f82a75e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:23.953474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64e977-a0c7-41e0-8b39-457f82a75e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:23.953762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64e977-a0c7-41e0-8b39-457f82a75e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:23.954041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64e977-a0c7-41e0-8b39-457f82a75e63 admin admin] PATH_INFO: `/v3/projects/603e569b8c654f569672037ef13391f6/users/a070766ef74348d5a7f353ed53bf5f3e/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:23.976265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64e977-a0c7-41e0-8b39-457f82a75e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=603e569b8c654f569672037ef13391f6, user_id=a070766ef74348d5a7f353ed53bf5f3e, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:23.979384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64e977-a0c7-41e0-8b39-457f82a75e63 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:24.005043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 954/1912] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:23 2022] PUT /identity/v3/projects/603e569b8c654f569672037ef13391f6/users/a070766ef74348d5a7f353ed53bf5f3e/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:24.017745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2d036-3b72-4d22-a267-25a175defc29 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:24.018026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2d036-3b72-4d22-a267-25a175defc29 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:24.018333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2d036-3b72-4d22-a267-25a175defc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:24.048076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-73d2d036-3b72-4d22-a267-25a175defc29 None None] MFA Rules not processed for user `a070766ef74348d5a7f353ed53bf5f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:24.049313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-73d2d036-3b72-4d22-a267-25a175defc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:24.093441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:24.117673 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 959/1913] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:24.129282 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64e977-a0c7-41e0-8b39-457f82a75e63 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:24.131080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-fe64e977-a0c7-41e0-8b39-457f82a75e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:24.176453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:24.189778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64e977-a0c7-41e0-8b39-457f82a75e63 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:24.194402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004c716-0b1b-464a-8a23-076da36e71b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:24.195282 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c716-0b1b-464a-8a23-076da36e71b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:24.195641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c716-0b1b-464a-8a23-076da36e71b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:24.195938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c716-0b1b-464a-8a23-076da36e71b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:24.205252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004c716-0b1b-464a-8a23-076da36e71b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:24.206741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004c716-0b1b-464a-8a23-076da36e71b4 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:24.215787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 955/1914] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:25.264542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2d036-3b72-4d22-a267-25a175defc29 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:25.265948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-73d2d036-3b72-4d22-a267-25a175defc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:25.330091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2d036-3b72-4d22-a267-25a175defc29 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:25.335137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:25.336065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:25.336349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:25.336628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] PATH_INFO: `/v3/users/d54c436b2634460caa5d11a9868e9095` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:25.338049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d54c436b2634460caa5d11a9868e9095)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:25.340377 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:25.397223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd54c436b2634460caa5d11a9868e9095'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:25.404385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd54c436b2634460caa5d11a9868e9095'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:25.405142 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd54c436b2634460caa5d11a9868e9095'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:25.405745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd54c436b2634460caa5d11a9868e9095'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:25.419080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd54c436b2634460caa5d11a9868e9095'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:25.432983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 960/1915] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:25 2022] DELETE /identity/v3/users/d54c436b2634460caa5d11a9868e9095 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:25.437493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ad47-3c2b-4918-82bd-e65a623876bf None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:25.437957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ad47-3c2b-4918-82bd-e65a623876bf None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:25.438075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ad47-3c2b-4918-82bd-e65a623876bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:25.457043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-8116ad47-3c2b-4918-82bd-e65a623876bf None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:25.458940 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8116ad47-3c2b-4918-82bd-e65a623876bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:25.524230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 956/1916] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:01:25.533739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:25.535226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:25.600380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafa2fa-7d47-4792-90c4-ec4cda0368c3 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:25.605166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e941cc-99ce-4693-8e40-8b03ecd923f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:25.605827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e941cc-99ce-4693-8e40-8b03ecd923f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:25.606189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e941cc-99ce-4693-8e40-8b03ecd923f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:25.606415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e941cc-99ce-4693-8e40-8b03ecd923f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:25.616213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e941cc-99ce-4693-8e40-8b03ecd923f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:25.618404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e941cc-99ce-4693-8e40-8b03ecd923f7 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:25.628536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 961/1917] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:26.228105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116ad47-3c2b-4918-82bd-e65a623876bf None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:26.229798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8116ad47-3c2b-4918-82bd-e65a623876bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:26.255207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116ad47-3c2b-4918-82bd-e65a623876bf None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:26.278464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:26.284433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:26.284889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:26.285286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] PATH_INFO: `/v3/projects/e2bcca0447ec41b798421f242ec00ce2` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:26.290809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2bcca0447ec41b798421f242ec00ce2)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:26.293503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:26.429243 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e941cc-99ce-4693-8e40-8b03ecd923f7 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:26.431261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-59e941cc-99ce-4693-8e40-8b03ecd923f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:26.435002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2bcca0447ec41b798421f242ec00ce2', 'request_id': 'req-4a88b875-a556-4dea-b782-427397336070'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.446530 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:26.450375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2bcca0447ec41b798421f242ec00ce2', 'request_id': 'req-4a88b875-a556-4dea-b782-427397336070'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.460629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e941cc-99ce-4693-8e40-8b03ecd923f7 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:26.466613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2bcca0447ec41b798421f242ec00ce2', 'request_id': 'req-4a88b875-a556-4dea-b782-427397336070'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.505816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] The token cache is being invalidate because project e2bcca0447ec41b798421f242ec00ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:26.507455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.510388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.514181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 957/1918] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:26 2022] DELETE /identity/v3/projects/e2bcca0447ec41b798421f242ec00ce2 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:26.530776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:26.537248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:26.540875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3845d49-51b6-4f19-aa08-7edba895470f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:26.542004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3845d49-51b6-4f19-aa08-7edba895470f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:26.542527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3845d49-51b6-4f19-aa08-7edba895470f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:26.542983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3845d49-51b6-4f19-aa08-7edba895470f admin admin] PATH_INFO: `/v3/users/6dfe4b5e48d445ef803faf8935a62aba` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:26.546568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3845d49-51b6-4f19-aa08-7edba895470f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dfe4b5e48d445ef803faf8935a62aba)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:26.548770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3845d49-51b6-4f19-aa08-7edba895470f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:26.572101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b875-a556-4dea-b782-427397336070 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:26.579343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:26.580530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:26.581067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:26.581931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] PATH_INFO: `/v3/users/bcf52210bb4a43f5846b5da2efe36876` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:26.584960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf52210bb4a43f5846b5da2efe36876)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:26.587792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:26.619358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b3845d49-51b6-4f19-aa08-7edba895470f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dfe4b5e48d445ef803faf8935a62aba'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.628721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b3845d49-51b6-4f19-aa08-7edba895470f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dfe4b5e48d445ef803faf8935a62aba'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.630170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b3845d49-51b6-4f19-aa08-7edba895470f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dfe4b5e48d445ef803faf8935a62aba'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.631107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b3845d49-51b6-4f19-aa08-7edba895470f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dfe4b5e48d445ef803faf8935a62aba'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.646067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b3845d49-51b6-4f19-aa08-7edba895470f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dfe4b5e48d445ef803faf8935a62aba'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.659198 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf52210bb4a43f5846b5da2efe36876'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.667650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 962/1919] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:26 2022] DELETE /identity/v3/users/6dfe4b5e48d445ef803faf8935a62aba => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:01:26.669390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b1e5c-ee30-41eb-a2f4-29b6ec4ba87a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:26.670033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b1e5c-ee30-41eb-a2f4-29b6ec4ba87a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:26.670577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b1e5c-ee30-41eb-a2f4-29b6ec4ba87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:26.673646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:26.675664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf52210bb4a43f5846b5da2efe36876'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.687988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf52210bb4a43f5846b5da2efe36876'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.689298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf52210bb4a43f5846b5da2efe36876'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.697240 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-612b1e5c-ee30-41eb-a2f4-29b6ec4ba87a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:26.699305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-612b1e5c-ee30-41eb-a2f4-29b6ec4ba87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:26.700724 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf52210bb4a43f5846b5da2efe36876'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:26.708366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 958/1920] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:26 2022] DELETE /identity/v3/users/bcf52210bb4a43f5846b5da2efe36876 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:26.807391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 963/1921] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:26.816526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:26.818147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:26.911917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8b817-d1a1-40c7-a25b-6eb5aea1632e admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:26.918876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec8b02-6a6a-4417-8171-874565313efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:26.919668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec8b02-6a6a-4417-8171-874565313efd service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:26.919962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec8b02-6a6a-4417-8171-874565313efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:26.920242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec8b02-6a6a-4417-8171-874565313efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:26.982350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec8b02-6a6a-4417-8171-874565313efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:26.983801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec8b02-6a6a-4417-8171-874565313efd service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:26.993654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 959/1922] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:27.056457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b1e5c-ee30-41eb-a2f4-29b6ec4ba87a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:27.057917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-612b1e5c-ee30-41eb-a2f4-29b6ec4ba87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:27.070481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b1e5c-ee30-41eb-a2f4-29b6ec4ba87a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:27.075769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:27.076652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.077012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.077208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] PATH_INFO: `/v3/projects/a530fa9155dd449990dc061c5cde4747` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.078443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a530fa9155dd449990dc061c5cde4747)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:27.080400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:27.111694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:27.143534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a530fa9155dd449990dc061c5cde4747', 'request_id': 'req-a63cd35f-2892-461e-8450-bc9845bf35f6'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:27.171850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a530fa9155dd449990dc061c5cde4747', 'request_id': 'req-a63cd35f-2892-461e-8450-bc9845bf35f6'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:27.182695 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a530fa9155dd449990dc061c5cde4747', 'request_id': 'req-a63cd35f-2892-461e-8450-bc9845bf35f6'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:27.198914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] The token cache is being invalidate because project a530fa9155dd449990dc061c5cde4747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:27.199270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:27.200239 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a63cd35f-2892-461e-8450-bc9845bf35f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:27.202748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 964/1923] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:27 2022] DELETE /identity/v3/projects/a530fa9155dd449990dc061c5cde4747 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:27.207823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f01d8-f153-4c16-86c3-1b0aa7207e43 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.208183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f01d8-f153-4c16-86c3-1b0aa7207e43 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.208584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f01d8-f153-4c16-86c3-1b0aa7207e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.237182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-733f01d8-f153-4c16-86c3-1b0aa7207e43 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:27.238429 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-733f01d8-f153-4c16-86c3-1b0aa7207e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:27.350994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 960/1924] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:01:27.359214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23dbe6-5345-4610-a84e-1b4d8a3198e8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.359767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23dbe6-5345-4610-a84e-1b4d8a3198e8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.360294 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23dbe6-5345-4610-a84e-1b4d8a3198e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.373641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f01d8-f153-4c16-86c3-1b0aa7207e43 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:27.375238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-733f01d8-f153-4c16-86c3-1b0aa7207e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:27.397247 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f01d8-f153-4c16-86c3-1b0aa7207e43 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:27.399485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9e23dbe6-5345-4610-a84e-1b4d8a3198e8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:27.402441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9e23dbe6-5345-4610-a84e-1b4d8a3198e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:27.406625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:27.408135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.408527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.409057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] PATH_INFO: `/v3/projects/baa960629c264974bcbd3f64c6f04504` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.411498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baa960629c264974bcbd3f64c6f04504)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:27.413765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:27.420582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 965/1925] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:27.425694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23dbe6-5345-4610-a84e-1b4d8a3198e8 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:27.440972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23dbe6-5345-4610-a84e-1b4d8a3198e8 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:27.447625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3429c-6740-4403-a70d-5aea87caf401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:27.448942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3429c-6740-4403-a70d-5aea87caf401 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.449375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3429c-6740-4403-a70d-5aea87caf401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.450086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3429c-6740-4403-a70d-5aea87caf401 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.451489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3429c-6740-4403-a70d-5aea87caf401 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:01:27.452007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3429c-6740-4403-a70d-5aea87caf401 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:27.454099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3429c-6740-4403-a70d-5aea87caf401 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:27.472009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:27.488944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 966/1926] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:01:27.490672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baa960629c264974bcbd3f64c6f04504', 'request_id': 'req-4b58117f-7c78-4344-83c9-0f54848e1f32'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:27.493923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3429c-6740-4403-a70d-5aea87caf401 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:27.504503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baa960629c264974bcbd3f64c6f04504', 'request_id': 'req-4b58117f-7c78-4344-83c9-0f54848e1f32'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:27.510402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3429c-6740-4403-a70d-5aea87caf401 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:27.512675 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baa960629c264974bcbd3f64c6f04504', 'request_id': 'req-4b58117f-7c78-4344-83c9-0f54848e1f32'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:27.530439 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] The token cache is being invalidate because project baa960629c264974bcbd3f64c6f04504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:27.530933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:27.531790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4b58117f-7c78-4344-83c9-0f54848e1f32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:27.533873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 961/1927] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:27 2022] DELETE /identity/v3/projects/baa960629c264974bcbd3f64c6f04504 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:01:27.539973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bc642-6f9e-4ba1-ae3f-fbc24ef3c850 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.540435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bc642-6f9e-4ba1-ae3f-fbc24ef3c850 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.540928 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bc642-6f9e-4ba1-ae3f-fbc24ef3c850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.565658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-196bc642-6f9e-4ba1-ae3f-fbc24ef3c850 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:27.566935 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-196bc642-6f9e-4ba1-ae3f-fbc24ef3c850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:27.584128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc70e5-c15f-487f-b68b-c82dbfc93aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:27.584991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc70e5-c15f-487f-b68b-c82dbfc93aac admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.585869 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc70e5-c15f-487f-b68b-c82dbfc93aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.586369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc70e5-c15f-487f-b68b-c82dbfc93aac admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.587335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc70e5-c15f-487f-b68b-c82dbfc93aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:27.589626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc70e5-c15f-487f-b68b-c82dbfc93aac admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:27.625261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 967/1928] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:27 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:27.629355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc70e5-c15f-487f-b68b-c82dbfc93aac admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:27.631156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5ccc70e5-c15f-487f-b68b-c82dbfc93aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:27.650833 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 962/1929] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:27.659681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9da6fa-7419-4f7d-8a01-f7acc6e4894f None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.660037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9da6fa-7419-4f7d-8a01-f7acc6e4894f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.660385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9da6fa-7419-4f7d-8a01-f7acc6e4894f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.680118 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-4d9da6fa-7419-4f7d-8a01-f7acc6e4894f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:27.681527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4d9da6fa-7419-4f7d-8a01-f7acc6e4894f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:27.708413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc70e5-c15f-487f-b68b-c82dbfc93aac admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:27.714995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873d8e9-306b-483f-8bc5-63913082f5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:27.715684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873d8e9-306b-483f-8bc5-63913082f5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.715974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873d8e9-306b-483f-8bc5-63913082f5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.716249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873d8e9-306b-483f-8bc5-63913082f5f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.716951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873d8e9-306b-483f-8bc5-63913082f5f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:27.719739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873d8e9-306b-483f-8bc5-63913082f5f7 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:27.745012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 968/1930] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:27 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:01:27.748946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873d8e9-306b-483f-8bc5-63913082f5f7 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:27.753942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:27.760012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873d8e9-306b-483f-8bc5-63913082f5f7 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:27.764117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 963/1931] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:01:27.767819 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa1737-cabb-44dd-b035-1c10a2a7f40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:27.768544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9da6fa-7419-4f7d-8a01-f7acc6e4894f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:27.768694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa1737-cabb-44dd-b035-1c10a2a7f40b admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.769010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa1737-cabb-44dd-b035-1c10a2a7f40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.769739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa1737-cabb-44dd-b035-1c10a2a7f40b admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.770574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa1737-cabb-44dd-b035-1c10a2a7f40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:27.772516 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa1737-cabb-44dd-b035-1c10a2a7f40b admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:27.779058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9da6fa-7419-4f7d-8a01-f7acc6e4894f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:27.784176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1804ef3-f570-46d2-a280-23ac021409b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:27.784742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 969/1932] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:01:27.786809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1804ef3-f570-46d2-a280-23ac021409b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.787103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1804ef3-f570-46d2-a280-23ac021409b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.787390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1804ef3-f570-46d2-a280-23ac021409b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.788132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1804ef3-f570-46d2-a280-23ac021409b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:01:27.788501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1804ef3-f570-46d2-a280-23ac021409b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:27.790121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1804ef3-f570-46d2-a280-23ac021409b3 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:27.791243 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa1737-cabb-44dd-b035-1c10a2a7f40b admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:27.802321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 964/1933] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:01:27.803336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa1737-cabb-44dd-b035-1c10a2a7f40b admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:27.806416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1804ef3-f570-46d2-a280-23ac021409b3 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:27.808392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154ddcb-9865-4252-a7d8-8e600891cb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:27.809049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154ddcb-9865-4252-a7d8-8e600891cb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.809334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154ddcb-9865-4252-a7d8-8e600891cb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.809607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154ddcb-9865-4252-a7d8-8e600891cb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.810486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154ddcb-9865-4252-a7d8-8e600891cb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:27.812078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154ddcb-9865-4252-a7d8-8e600891cb10 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:27.816258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1804ef3-f570-46d2-a280-23ac021409b3 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:27.821440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661a61b-5578-4f90-b39b-afb15ce4fc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:27.822072 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661a61b-5578-4f90-b39b-afb15ce4fc5e admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.822463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661a61b-5578-4f90-b39b-afb15ce4fc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.822753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661a61b-5578-4f90-b39b-afb15ce4fc5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.823463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661a61b-5578-4f90-b39b-afb15ce4fc5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:27.823582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 970/1934] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:27.825504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661a61b-5578-4f90-b39b-afb15ce4fc5e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:27.827412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154ddcb-9865-4252-a7d8-8e600891cb10 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:27.837718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154ddcb-9865-4252-a7d8-8e600891cb10 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:27.842649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c9ee9-01ff-491e-b55a-aeb3441db5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:27.843499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c9ee9-01ff-491e-b55a-aeb3441db5e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.843847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c9ee9-01ff-491e-b55a-aeb3441db5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.844165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c9ee9-01ff-491e-b55a-aeb3441db5e3 admin admin] PATH_INFO: `/v3/projects/f99fa8a74fdb4c5ab9ffe038301db2a9/users/68c1ecd3d5f6421cabe0654ae61a866c/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.852806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 965/1935] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:27 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:01:27.857325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661a61b-5578-4f90-b39b-afb15ce4fc5e admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:27.859124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c9ee9-01ff-491e-b55a-aeb3441db5e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99fa8a74fdb4c5ab9ffe038301db2a9, user_id=68c1ecd3d5f6421cabe0654ae61a866c, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:27.861864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c9ee9-01ff-491e-b55a-aeb3441db5e3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:27.877113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 971/1936] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:27 2022] PUT /identity/v3/projects/f99fa8a74fdb4c5ab9ffe038301db2a9/users/68c1ecd3d5f6421cabe0654ae61a866c/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:01:27.882751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:27.888592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f8392-1252-40cd-b41a-1ea7acc79b50 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:27.888891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f8392-1252-40cd-b41a-1ea7acc79b50 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:27.889177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f8392-1252-40cd-b41a-1ea7acc79b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:27.920067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c55f8392-1252-40cd-b41a-1ea7acc79b50 None None] MFA Rules not processed for user `68c1ecd3d5f6421cabe0654ae61a866c`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:27.921333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c55f8392-1252-40cd-b41a-1ea7acc79b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:27.938400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661a61b-5578-4f90-b39b-afb15ce4fc5e admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:27.995687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 972/1937] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:01:28.000690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f8392-1252-40cd-b41a-1ea7acc79b50 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.015290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94618c6-bf3e-4be5-b3c3-79c3e1f7a282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.015978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94618c6-bf3e-4be5-b3c3-79c3e1f7a282 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.016268 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94618c6-bf3e-4be5-b3c3-79c3e1f7a282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.016549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94618c6-bf3e-4be5-b3c3-79c3e1f7a282 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.017287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94618c6-bf3e-4be5-b3c3-79c3e1f7a282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.019726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94618c6-bf3e-4be5-b3c3-79c3e1f7a282 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.044723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 966/1938] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:27 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:01:28.048636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94618c6-bf3e-4be5-b3c3-79c3e1f7a282 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.058709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94618c6-bf3e-4be5-b3c3-79c3e1f7a282 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.063537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad8df3-c9bf-4e9a-b9e0-76b4e0769f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.064198 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad8df3-c9bf-4e9a-b9e0-76b4e0769f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.064488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad8df3-c9bf-4e9a-b9e0-76b4e0769f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.064771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad8df3-c9bf-4e9a-b9e0-76b4e0769f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.065540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad8df3-c9bf-4e9a-b9e0-76b4e0769f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.067174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad8df3-c9bf-4e9a-b9e0-76b4e0769f50 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.070373 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f8392-1252-40cd-b41a-1ea7acc79b50 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.075505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bc6c2-3936-45f2-8e8f-748fa19f6f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.076190 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bc6c2-3936-45f2-8e8f-748fa19f6f07 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.076481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bc6c2-3936-45f2-8e8f-748fa19f6f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.076750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bc6c2-3936-45f2-8e8f-748fa19f6f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.077444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bc6c2-3936-45f2-8e8f-748fa19f6f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.077887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 967/1939] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:01:28.079410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bc6c2-3936-45f2-8e8f-748fa19f6f07 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.081587 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad8df3-c9bf-4e9a-b9e0-76b4e0769f50 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.092556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad8df3-c9bf-4e9a-b9e0-76b4e0769f50 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.097721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505ef43-e680-41ab-a8fd-35c3991d84b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.098453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505ef43-e680-41ab-a8fd-35c3991d84b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.098745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505ef43-e680-41ab-a8fd-35c3991d84b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.099027 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505ef43-e680-41ab-a8fd-35c3991d84b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.099902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505ef43-e680-41ab-a8fd-35c3991d84b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.101510 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505ef43-e680-41ab-a8fd-35c3991d84b0 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.106767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 973/1940] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:27 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:01:28.110396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bc6c2-3936-45f2-8e8f-748fa19f6f07 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.118035 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 968/1941] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:01:28.119046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505ef43-e680-41ab-a8fd-35c3991d84b0 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.129229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505ef43-e680-41ab-a8fd-35c3991d84b0 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.143203 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:28.174842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bc6c2-3936-45f2-8e8f-748fa19f6f07 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.180389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1e39c-efc7-4a44-a35f-1794afa71460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.181096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1e39c-efc7-4a44-a35f-1794afa71460 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.181382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1e39c-efc7-4a44-a35f-1794afa71460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.181689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1e39c-efc7-4a44-a35f-1794afa71460 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.182469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1e39c-efc7-4a44-a35f-1794afa71460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.184466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1e39c-efc7-4a44-a35f-1794afa71460 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.189645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b175ed-2d49-4aa9-88a9-2810cee16e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.190586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b175ed-2d49-4aa9-88a9-2810cee16e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.190874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b175ed-2d49-4aa9-88a9-2810cee16e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.191154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b175ed-2d49-4aa9-88a9-2810cee16e38 admin admin] PATH_INFO: `/v3/projects/6f2aa286c68c473686a4c71190684e18/users/226c8c0918714060ac908316c7effc08/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.206200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 974/1942] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:28 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:01:28.207252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b175ed-2d49-4aa9-88a9-2810cee16e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2aa286c68c473686a4c71190684e18, user_id=226c8c0918714060ac908316c7effc08, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.209921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b175ed-2d49-4aa9-88a9-2810cee16e38 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.210043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1e39c-efc7-4a44-a35f-1794afa71460 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.220139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1e39c-efc7-4a44-a35f-1794afa71460 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.227005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 969/1943] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:28 2022] PUT /identity/v3/projects/6f2aa286c68c473686a4c71190684e18/users/226c8c0918714060ac908316c7effc08/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:01:28.232918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d55b3a-dc6b-4bda-a2bd-765be0d9bdfd None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.233980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d55b3a-dc6b-4bda-a2bd-765be0d9bdfd None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.234265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d55b3a-dc6b-4bda-a2bd-765be0d9bdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.252526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-84d55b3a-dc6b-4bda-a2bd-765be0d9bdfd None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:28.253746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-84d55b3a-dc6b-4bda-a2bd-765be0d9bdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:28.268762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:28.291176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece3c60-4b8c-42fd-8081-0f1eed213566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.291886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece3c60-4b8c-42fd-8081-0f1eed213566 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.292174 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece3c60-4b8c-42fd-8081-0f1eed213566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.292450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece3c60-4b8c-42fd-8081-0f1eed213566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.293209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece3c60-4b8c-42fd-8081-0f1eed213566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.294934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece3c60-4b8c-42fd-8081-0f1eed213566 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.305123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 975/1944] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:01:28.309275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece3c60-4b8c-42fd-8081-0f1eed213566 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.318854 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece3c60-4b8c-42fd-8081-0f1eed213566 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.320993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 970/1945] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:01:28.324197 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdcfa8-71fb-4d9f-bb13-555ee5e3d01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.325071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdcfa8-71fb-4d9f-bb13-555ee5e3d01d admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.325356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdcfa8-71fb-4d9f-bb13-555ee5e3d01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.325633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdcfa8-71fb-4d9f-bb13-555ee5e3d01d admin admin] PATH_INFO: `/v3/projects/358cd22a40ab48f99d6f8f4d5d1009c4/users/5bc8a006ac1143d7b391ff32ddfaac21/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.330961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d55b3a-dc6b-4bda-a2bd-765be0d9bdfd None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.332322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-84d55b3a-dc6b-4bda-a2bd-765be0d9bdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:28.341178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fdcfa8-71fb-4d9f-bb13-555ee5e3d01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=358cd22a40ab48f99d6f8f4d5d1009c4, user_id=5bc8a006ac1143d7b391ff32ddfaac21, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.343972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fdcfa8-71fb-4d9f-bb13-555ee5e3d01d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.356926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 976/1946] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:28 2022] PUT /identity/v3/projects/358cd22a40ab48f99d6f8f4d5d1009c4/users/5bc8a006ac1143d7b391ff32ddfaac21/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:01:28.360693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdcfa8-71fb-4d9f-bb13-555ee5e3d01d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.392477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d55b3a-dc6b-4bda-a2bd-765be0d9bdfd None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.440356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdcfa8-71fb-4d9f-bb13-555ee5e3d01d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.445791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc1000-5504-45ba-938b-62ecdc9d791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.446645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc1000-5504-45ba-938b-62ecdc9d791a admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.446770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc1000-5504-45ba-938b-62ecdc9d791a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.447039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc1000-5504-45ba-938b-62ecdc9d791a admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.447777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc1000-5504-45ba-938b-62ecdc9d791a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.449542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc1000-5504-45ba-938b-62ecdc9d791a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.454971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb54286-3f36-4ef8-833d-b3509a488064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.455631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb54286-3f36-4ef8-833d-b3509a488064 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.455913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb54286-3f36-4ef8-833d-b3509a488064 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.456190 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb54286-3f36-4ef8-833d-b3509a488064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.462797 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 977/1947] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:01:28.463703 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc1000-5504-45ba-938b-62ecdc9d791a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.465907 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb54286-3f36-4ef8-833d-b3509a488064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.467378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb54286-3f36-4ef8-833d-b3509a488064 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.473426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc1000-5504-45ba-938b-62ecdc9d791a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.477100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 971/1948] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:01:28.479071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0c63e-076a-4535-996e-501a42fca25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.479968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0c63e-076a-4535-996e-501a42fca25a admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.480245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0c63e-076a-4535-996e-501a42fca25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.480526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0c63e-076a-4535-996e-501a42fca25a admin admin] PATH_INFO: `/v3/projects/358cd22a40ab48f99d6f8f4d5d1009c4/users/5bc8a006ac1143d7b391ff32ddfaac21/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.482558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0c63e-076a-4535-996e-501a42fca25a admin admin] RBAC: Authorizing `identity:create_grant(project_id=358cd22a40ab48f99d6f8f4d5d1009c4, user_id=5bc8a006ac1143d7b391ff32ddfaac21, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.485265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0c63e-076a-4535-996e-501a42fca25a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.497180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 978/1949] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:28 2022] PUT /identity/v3/projects/358cd22a40ab48f99d6f8f4d5d1009c4/users/5bc8a006ac1143d7b391ff32ddfaac21/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:01:28.507226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d4ffe-984c-4221-8c35-bc470e2e0411 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.508059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d4ffe-984c-4221-8c35-bc470e2e0411 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.508373 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d4ffe-984c-4221-8c35-bc470e2e0411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.539910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-2f8d4ffe-984c-4221-8c35-bc470e2e0411 None None] MFA Rules not processed for user `5bc8a006ac1143d7b391ff32ddfaac21`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:28.541314 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2f8d4ffe-984c-4221-8c35-bc470e2e0411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:28.624929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 972/1950] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:28 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:28.637380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0c63e-076a-4535-996e-501a42fca25a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.639627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-77a0c63e-076a-4535-996e-501a42fca25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:28.662862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:28.717229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0c63e-076a-4535-996e-501a42fca25a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.724221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53638-32fc-4016-b59f-999ee6d42762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.724221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53638-32fc-4016-b59f-999ee6d42762 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.724644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53638-32fc-4016-b59f-999ee6d42762 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.725544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53638-32fc-4016-b59f-999ee6d42762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.738472 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53638-32fc-4016-b59f-999ee6d42762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.740524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53638-32fc-4016-b59f-999ee6d42762 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.752446 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 979/1951] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:28 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:28.833507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d4ffe-984c-4221-8c35-bc470e2e0411 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:28.839491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:28.845902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d4ffe-984c-4221-8c35-bc470e2e0411 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:28.853875 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1131ee-29c2-4c3b-b740-fc7a979c4e51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:28.854781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1131ee-29c2-4c3b-b740-fc7a979c4e51 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:28.855083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1131ee-29c2-4c3b-b740-fc7a979c4e51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:28.855366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1131ee-29c2-4c3b-b740-fc7a979c4e51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:28.945141 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1131ee-29c2-4c3b-b740-fc7a979c4e51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:28.948004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1131ee-29c2-4c3b-b740-fc7a979c4e51 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:28.969045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 973/1952] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:01:29.302453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd5ecb-7986-4fff-b642-3e7227425c75 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:29.302963 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd5ecb-7986-4fff-b642-3e7227425c75 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:29.303250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd5ecb-7986-4fff-b642-3e7227425c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:29.355781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-08fd5ecb-7986-4fff-b642-3e7227425c75 None None] MFA Rules not processed for user `226c8c0918714060ac908316c7effc08`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:29.357007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-08fd5ecb-7986-4fff-b642-3e7227425c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:29.448424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 980/1953] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:29.458883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1131ee-29c2-4c3b-b740-fc7a979c4e51 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:29.460257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ee1131ee-29c2-4c3b-b740-fc7a979c4e51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:29.494092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1131ee-29c2-4c3b-b740-fc7a979c4e51 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:29.498158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097dc1e-7ba1-45f3-abf3-6e38d443d718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:29.498843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097dc1e-7ba1-45f3-abf3-6e38d443d718 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:29.499129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097dc1e-7ba1-45f3-abf3-6e38d443d718 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:29.499409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097dc1e-7ba1-45f3-abf3-6e38d443d718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:29.507721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097dc1e-7ba1-45f3-abf3-6e38d443d718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:29.509143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097dc1e-7ba1-45f3-abf3-6e38d443d718 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:29.517959 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 974/1954] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:32.963579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2376b-6bc1-4a0b-bb4e-656bd87f21a5 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:32.964084 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2376b-6bc1-4a0b-bb4e-656bd87f21a5 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:32.964175 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2376b-6bc1-4a0b-bb4e-656bd87f21a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:32.981441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-59c2376b-6bc1-4a0b-bb4e-656bd87f21a5 None None] MFA Rules not processed for user `226c8c0918714060ac908316c7effc08`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:32.982747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-59c2376b-6bc1-4a0b-bb4e-656bd87f21a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:32.989578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 981/1955] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:39.513953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=42 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:40.703054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097dc1e-7ba1-45f3-abf3-6e38d443d718 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:40.705291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6097dc1e-7ba1-45f3-abf3-6e38d443d718 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:40.864568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097dc1e-7ba1-45f3-abf3-6e38d443d718 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:40.879976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:40.880951 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:40.881310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:40.881661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] PATH_INFO: `/v3/users/1a15083ca15a4a32be4bc439655102b2` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:40.884818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a15083ca15a4a32be4bc439655102b2)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:40.886962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:40.972294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a15083ca15a4a32be4bc439655102b2'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.011497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a15083ca15a4a32be4bc439655102b2'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.020912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a15083ca15a4a32be4bc439655102b2'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.021572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a15083ca15a4a32be4bc439655102b2'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.030528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a15083ca15a4a32be4bc439655102b2'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.039216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 975/1956] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:40 2022] DELETE /identity/v3/users/1a15083ca15a4a32be4bc439655102b2 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:41.044268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72817110-0347-4458-a3be-4d50a3375cd4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:41.044746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72817110-0347-4458-a3be-4d50a3375cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:41.045144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72817110-0347-4458-a3be-4d50a3375cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:41.076662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-72817110-0347-4458-a3be-4d50a3375cd4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:41.077978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-72817110-0347-4458-a3be-4d50a3375cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:41.146619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 982/1957] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:41.156673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:41.158220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:41.222641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:41.263769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18d55c-0841-4051-bd27-b2051cd43706 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:41.268892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54221beb-aaa8-4358-b4c0-d20d0026137c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:41.270430 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54221beb-aaa8-4358-b4c0-d20d0026137c service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:41.270896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54221beb-aaa8-4358-b4c0-d20d0026137c service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:41.271522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54221beb-aaa8-4358-b4c0-d20d0026137c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:41.286535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54221beb-aaa8-4358-b4c0-d20d0026137c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:41.288509 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54221beb-aaa8-4358-b4c0-d20d0026137c service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:41.312299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 976/1958] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:41.686518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72817110-0347-4458-a3be-4d50a3375cd4 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:41.688055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-72817110-0347-4458-a3be-4d50a3375cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:41.700836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72817110-0347-4458-a3be-4d50a3375cd4 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:41.705605 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:41.706712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:41.707002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:41.707280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] PATH_INFO: `/v3/projects/7cd9be447af844249a5a696ceeaa4d23` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:41.708471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cd9be447af844249a5a696ceeaa4d23)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:41.711076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:41.713721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:41.759604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cd9be447af844249a5a696ceeaa4d23', 'request_id': 'req-a1b31230-2626-413f-87ce-8ea581252cdb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.767015 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cd9be447af844249a5a696ceeaa4d23', 'request_id': 'req-a1b31230-2626-413f-87ce-8ea581252cdb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.773546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cd9be447af844249a5a696ceeaa4d23', 'request_id': 'req-a1b31230-2626-413f-87ce-8ea581252cdb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.784499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] The token cache is being invalidate because project 7cd9be447af844249a5a696ceeaa4d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:41.784717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.785359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.787056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 983/1959] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:41 2022] DELETE /identity/v3/projects/7cd9be447af844249a5a696ceeaa4d23 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:41.791129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54221beb-aaa8-4358-b4c0-d20d0026137c service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:41.801035 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54221beb-aaa8-4358-b4c0-d20d0026137c service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:41.868474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0870b-a68a-4762-b33d-c99045c3f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:41.869650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0870b-a68a-4762-b33d-c99045c3f052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:41.870106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0870b-a68a-4762-b33d-c99045c3f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:41.870580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0870b-a68a-4762-b33d-c99045c3f052 admin admin] PATH_INFO: `/v3/users/8cb35404697642a991f039e65dcf32cc` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:41.872316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0870b-a68a-4762-b33d-c99045c3f052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb35404697642a991f039e65dcf32cc)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:41.874540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0870b-a68a-4762-b33d-c99045c3f052 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:41.934937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-45b0870b-a68a-4762-b33d-c99045c3f052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb35404697642a991f039e65dcf32cc'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.948244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-45b0870b-a68a-4762-b33d-c99045c3f052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb35404697642a991f039e65dcf32cc'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.958884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-45b0870b-a68a-4762-b33d-c99045c3f052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb35404697642a991f039e65dcf32cc'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.959583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-45b0870b-a68a-4762-b33d-c99045c3f052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb35404697642a991f039e65dcf32cc'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.968070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-45b0870b-a68a-4762-b33d-c99045c3f052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb35404697642a991f039e65dcf32cc'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:41.974304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 977/1960] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:41 2022] DELETE /identity/v3/users/8cb35404697642a991f039e65dcf32cc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:42.309455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:42.312340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:42.354921 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b31230-2626-413f-87ce-8ea581252cdb admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:42.426124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:42.427030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:42.427317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:42.427613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] PATH_INFO: `/v3/projects/ddb7900e23be47869eee7ae6a0b2c844` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:42.428818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb7900e23be47869eee7ae6a0b2c844)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:42.431250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:42.491206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb7900e23be47869eee7ae6a0b2c844', 'request_id': 'req-374e2738-7341-4edd-ad65-2fe31a489254'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:42.504159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb7900e23be47869eee7ae6a0b2c844', 'request_id': 'req-374e2738-7341-4edd-ad65-2fe31a489254'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:42.511895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb7900e23be47869eee7ae6a0b2c844', 'request_id': 'req-374e2738-7341-4edd-ad65-2fe31a489254'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:42.526633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] The token cache is being invalidate because project ddb7900e23be47869eee7ae6a0b2c844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:42.527004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:42.527801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-374e2738-7341-4edd-ad65-2fe31a489254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:42.536257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 984/1961] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:42 2022] DELETE /identity/v3/projects/ddb7900e23be47869eee7ae6a0b2c844 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:42.541117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde31995-e0cf-4704-a3f9-b67896aa2562 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:42.541347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde31995-e0cf-4704-a3f9-b67896aa2562 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:42.541628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde31995-e0cf-4704-a3f9-b67896aa2562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:42.589851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-fde31995-e0cf-4704-a3f9-b67896aa2562 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:42.591110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-fde31995-e0cf-4704-a3f9-b67896aa2562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:42.608226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:42.655338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 978/1962] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:01:42.662243 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9db484-6f2b-4cb9-904f-34fb1a4a32e2 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:42.662565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9db484-6f2b-4cb9-904f-34fb1a4a32e2 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:42.662847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9db484-6f2b-4cb9-904f-34fb1a4a32e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:42.692958 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-2c9db484-6f2b-4cb9-904f-34fb1a4a32e2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:42.694382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2c9db484-6f2b-4cb9-904f-34fb1a4a32e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:42.703127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 985/1963] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:42.707199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde31995-e0cf-4704-a3f9-b67896aa2562 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:42.716794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde31995-e0cf-4704-a3f9-b67896aa2562 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:42.721477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fca2a-b17f-4295-b3f9-7068e6fdd738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:42.722197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fca2a-b17f-4295-b3f9-7068e6fdd738 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:42.722516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fca2a-b17f-4295-b3f9-7068e6fdd738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:42.722790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fca2a-b17f-4295-b3f9-7068e6fdd738 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:42.723520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fca2a-b17f-4295-b3f9-7068e6fdd738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:01:42.723826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fca2a-b17f-4295-b3f9-7068e6fdd738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:42.725409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fca2a-b17f-4295-b3f9-7068e6fdd738 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:42.738205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 979/1964] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:01:42.742929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9db484-6f2b-4cb9-904f-34fb1a4a32e2 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:42.764984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9db484-6f2b-4cb9-904f-34fb1a4a32e2 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:42.775094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f79ae6-10d6-43dc-befd-9beeefcac8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:42.775862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f79ae6-10d6-43dc-befd-9beeefcac8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:42.776155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f79ae6-10d6-43dc-befd-9beeefcac8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:42.776436 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f79ae6-10d6-43dc-befd-9beeefcac8a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:42.777155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f79ae6-10d6-43dc-befd-9beeefcac8a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:42.779490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f79ae6-10d6-43dc-befd-9beeefcac8a8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:42.794885 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:42.812124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 986/1965] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:42 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:01:42.816444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fca2a-b17f-4295-b3f9-7068e6fdd738 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:42.830956 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fca2a-b17f-4295-b3f9-7068e6fdd738 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:42.897793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de85b1-0fcd-4436-880b-69cf333f14df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:42.898536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de85b1-0fcd-4436-880b-69cf333f14df admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:42.898823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de85b1-0fcd-4436-880b-69cf333f14df admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:42.899104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de85b1-0fcd-4436-880b-69cf333f14df admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:42.899800 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de85b1-0fcd-4436-880b-69cf333f14df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:42.901774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de85b1-0fcd-4436-880b-69cf333f14df admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:42.922888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 980/1966] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:42 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:01:42.927001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f79ae6-10d6-43dc-befd-9beeefcac8a8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:42.938090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f79ae6-10d6-43dc-befd-9beeefcac8a8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:42.942847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d52b7e-05da-4e3e-b689-d96ff5e210ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:42.943522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d52b7e-05da-4e3e-b689-d96ff5e210ac admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:42.943830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d52b7e-05da-4e3e-b689-d96ff5e210ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:42.944113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d52b7e-05da-4e3e-b689-d96ff5e210ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:42.945144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d52b7e-05da-4e3e-b689-d96ff5e210ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:42.946817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d52b7e-05da-4e3e-b689-d96ff5e210ac admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:42.958354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 987/1967] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:01:42.962555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de85b1-0fcd-4436-880b-69cf333f14df admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:42.974972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de85b1-0fcd-4436-880b-69cf333f14df admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:42.981511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3e8f0-df96-4202-bfaf-24f63cef5a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:42.982185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3e8f0-df96-4202-bfaf-24f63cef5a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:42.982509 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3e8f0-df96-4202-bfaf-24f63cef5a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:42.982785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3e8f0-df96-4202-bfaf-24f63cef5a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:42.983509 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3e8f0-df96-4202-bfaf-24f63cef5a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:42.985289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3e8f0-df96-4202-bfaf-24f63cef5a99 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:42.997719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 981/1968] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:01:43.001717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d52b7e-05da-4e3e-b689-d96ff5e210ac admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:43.013918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d52b7e-05da-4e3e-b689-d96ff5e210ac admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:43.019111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d35147-e353-4310-bdcf-39520a1e311c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:43.020014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d35147-e353-4310-bdcf-39520a1e311c admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:43.020301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d35147-e353-4310-bdcf-39520a1e311c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:43.020582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d35147-e353-4310-bdcf-39520a1e311c admin admin] PATH_INFO: `/v3/projects/665e8ebc26394f92befa75a61f90e792/users/cbfed2248f8344b7959ee7ee41f2e3f5/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:43.039143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d35147-e353-4310-bdcf-39520a1e311c admin admin] RBAC: Authorizing `identity:create_grant(project_id=665e8ebc26394f92befa75a61f90e792, user_id=cbfed2248f8344b7959ee7ee41f2e3f5, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:43.041863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d35147-e353-4310-bdcf-39520a1e311c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:43.055563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 988/1969] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:43 2022] PUT /identity/v3/projects/665e8ebc26394f92befa75a61f90e792/users/cbfed2248f8344b7959ee7ee41f2e3f5/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:01:43.066041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4902311-9c4b-4349-973c-0ca0b5948fac None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:43.066367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4902311-9c4b-4349-973c-0ca0b5948fac None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:43.066654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4902311-9c4b-4349-973c-0ca0b5948fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:43.097066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e4902311-9c4b-4349-973c-0ca0b5948fac None None] MFA Rules not processed for user `cbfed2248f8344b7959ee7ee41f2e3f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:43.098569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e4902311-9c4b-4349-973c-0ca0b5948fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:43.170997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 982/1970] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:43.175417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d35147-e353-4310-bdcf-39520a1e311c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:43.237407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d35147-e353-4310-bdcf-39520a1e311c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:43.242177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b863f4-5e96-4657-b9e3-0e4bd20d427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:43.242877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b863f4-5e96-4657-b9e3-0e4bd20d427c admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:43.243159 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b863f4-5e96-4657-b9e3-0e4bd20d427c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:43.243429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b863f4-5e96-4657-b9e3-0e4bd20d427c admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:43.244132 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b863f4-5e96-4657-b9e3-0e4bd20d427c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:43.246344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b863f4-5e96-4657-b9e3-0e4bd20d427c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:43.275612 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 989/1971] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:43 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:43.276824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4902311-9c4b-4349-973c-0ca0b5948fac None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:43.287911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4902311-9c4b-4349-973c-0ca0b5948fac None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:43.301068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:43.346215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82107a5e-7667-4f97-9ff4-d222682a1300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:43.346927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82107a5e-7667-4f97-9ff4-d222682a1300 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:43.347263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82107a5e-7667-4f97-9ff4-d222682a1300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:43.347547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82107a5e-7667-4f97-9ff4-d222682a1300 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:43.348300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82107a5e-7667-4f97-9ff4-d222682a1300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:43.350385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82107a5e-7667-4f97-9ff4-d222682a1300 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:43.381683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 983/1972] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:43 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:01:43.387476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b863f4-5e96-4657-b9e3-0e4bd20d427c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:43.395081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b863f4-5e96-4657-b9e3-0e4bd20d427c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:43.408371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80fce8-78f5-4cb7-9e6a-58685ba7a829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:43.409386 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80fce8-78f5-4cb7-9e6a-58685ba7a829 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:43.409838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80fce8-78f5-4cb7-9e6a-58685ba7a829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:43.410120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80fce8-78f5-4cb7-9e6a-58685ba7a829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:43.410926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80fce8-78f5-4cb7-9e6a-58685ba7a829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:43.412597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80fce8-78f5-4cb7-9e6a-58685ba7a829 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:43.426964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 990/1973] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:43.431371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82107a5e-7667-4f97-9ff4-d222682a1300 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:43.441516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82107a5e-7667-4f97-9ff4-d222682a1300 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:43.454772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae5a81-54d1-431a-8ea4-b3118417640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:43.454772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae5a81-54d1-431a-8ea4-b3118417640c admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:43.454772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae5a81-54d1-431a-8ea4-b3118417640c admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:43.454772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae5a81-54d1-431a-8ea4-b3118417640c admin admin] PATH_INFO: `/v3/projects/489dd2d53f174e62baa25f0b53b1a6cc/users/19e34005a18a472b82a55ff84cced9c7/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:43.468698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae5a81-54d1-431a-8ea4-b3118417640c admin admin] RBAC: Authorizing `identity:create_grant(project_id=489dd2d53f174e62baa25f0b53b1a6cc, user_id=19e34005a18a472b82a55ff84cced9c7, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:43.471385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae5a81-54d1-431a-8ea4-b3118417640c admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:43.484932 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 984/1974] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:43 2022] PUT /identity/v3/projects/489dd2d53f174e62baa25f0b53b1a6cc/users/19e34005a18a472b82a55ff84cced9c7/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:43.488911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80fce8-78f5-4cb7-9e6a-58685ba7a829 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:43.500111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80fce8-78f5-4cb7-9e6a-58685ba7a829 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:43.565139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958eddb8-9ded-4d9b-8f80-678b9efcc0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:43.566068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958eddb8-9ded-4d9b-8f80-678b9efcc0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:43.566604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958eddb8-9ded-4d9b-8f80-678b9efcc0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:43.567676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958eddb8-9ded-4d9b-8f80-678b9efcc0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:43.568617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958eddb8-9ded-4d9b-8f80-678b9efcc0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:43.571106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958eddb8-9ded-4d9b-8f80-678b9efcc0cd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:43.582814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 991/1975] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:43.586926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae5a81-54d1-431a-8ea4-b3118417640c admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:43.605220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae5a81-54d1-431a-8ea4-b3118417640c admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:43.612032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be2bf8-6a43-4a81-9a80-3b6e1f09840a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:43.613127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be2bf8-6a43-4a81-9a80-3b6e1f09840a admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:43.613518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be2bf8-6a43-4a81-9a80-3b6e1f09840a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:43.613868 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be2bf8-6a43-4a81-9a80-3b6e1f09840a admin admin] PATH_INFO: `/v3/projects/489dd2d53f174e62baa25f0b53b1a6cc/users/19e34005a18a472b82a55ff84cced9c7/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:43.616102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be2bf8-6a43-4a81-9a80-3b6e1f09840a admin admin] RBAC: Authorizing `identity:create_grant(project_id=489dd2d53f174e62baa25f0b53b1a6cc, user_id=19e34005a18a472b82a55ff84cced9c7, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:43.619050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be2bf8-6a43-4a81-9a80-3b6e1f09840a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:43.631086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 985/1976] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:43 2022] PUT /identity/v3/projects/489dd2d53f174e62baa25f0b53b1a6cc/users/19e34005a18a472b82a55ff84cced9c7/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:43.641858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762ffa1-49ed-4d27-8d6f-7b30a8237e92 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:43.642180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762ffa1-49ed-4d27-8d6f-7b30a8237e92 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:43.642476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762ffa1-49ed-4d27-8d6f-7b30a8237e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:43.670979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7762ffa1-49ed-4d27-8d6f-7b30a8237e92 None None] MFA Rules not processed for user `19e34005a18a472b82a55ff84cced9c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:43.672191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7762ffa1-49ed-4d27-8d6f-7b30a8237e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:43.692421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:43.759321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 992/1977] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:43.771829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be2bf8-6a43-4a81-9a80-3b6e1f09840a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:43.774379 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b4be2bf8-6a43-4a81-9a80-3b6e1f09840a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:43.838642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be2bf8-6a43-4a81-9a80-3b6e1f09840a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:43.842971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ce706-37cf-44b0-8eee-555a8827cbbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:43.843676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ce706-37cf-44b0-8eee-555a8827cbbe service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:43.843968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ce706-37cf-44b0-8eee-555a8827cbbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:43.844252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ce706-37cf-44b0-8eee-555a8827cbbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:43.856937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ce706-37cf-44b0-8eee-555a8827cbbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:43.863334 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ce706-37cf-44b0-8eee-555a8827cbbe service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:43.872540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 986/1978] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:44.266544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762ffa1-49ed-4d27-8d6f-7b30a8237e92 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:44.275581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762ffa1-49ed-4d27-8d6f-7b30a8237e92 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:44.280200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd095a-f0a8-41ab-a786-96e374bc0b46 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e34005a18a472b82a55ff84cced9c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489dd2d53f174e62baa25f0b53b1a6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:44.280856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd095a-f0a8-41ab-a786-96e374bc0b46 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:44.281155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd095a-f0a8-41ab-a786-96e374bc0b46 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:44.281427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd095a-f0a8-41ab-a786-96e374bc0b46 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:44.282158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd095a-f0a8-41ab-a786-96e374bc0b46 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:01:44.282508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd095a-f0a8-41ab-a786-96e374bc0b46 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:44.284120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd095a-f0a8-41ab-a786-96e374bc0b46 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:44.296034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 993/1979] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:01:44.300266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ce706-37cf-44b0-8eee-555a8827cbbe service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:44.309921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ce706-37cf-44b0-8eee-555a8827cbbe service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:44.314631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318addc-9270-4697-b13c-f7e4a1d92d7c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e34005a18a472b82a55ff84cced9c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489dd2d53f174e62baa25f0b53b1a6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:44.315281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318addc-9270-4697-b13c-f7e4a1d92d7c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:44.315565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318addc-9270-4697-b13c-f7e4a1d92d7c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:44.315840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318addc-9270-4697-b13c-f7e4a1d92d7c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:44.316533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318addc-9270-4697-b13c-f7e4a1d92d7c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:44.318444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318addc-9270-4697-b13c-f7e4a1d92d7c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:44.346257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 987/1980] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:44 2022] POST /identity/v3/projects => generated 353 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:44.350698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd095a-f0a8-41ab-a786-96e374bc0b46 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:44.361691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd095a-f0a8-41ab-a786-96e374bc0b46 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:44.420431 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31d26d-1b12-4745-85eb-355a5f0e8f10 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e34005a18a472b82a55ff84cced9c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489dd2d53f174e62baa25f0b53b1a6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:44.421113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31d26d-1b12-4745-85eb-355a5f0e8f10 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:44.421412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31d26d-1b12-4745-85eb-355a5f0e8f10 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:44.421747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31d26d-1b12-4745-85eb-355a5f0e8f10 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:44.422523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31d26d-1b12-4745-85eb-355a5f0e8f10 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:01:44.422840 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31d26d-1b12-4745-85eb-355a5f0e8f10 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:44.424553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31d26d-1b12-4745-85eb-355a5f0e8f10 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:44.437991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 994/1981] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:01:44.480021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318addc-9270-4697-b13c-f7e4a1d92d7c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:44.490853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318addc-9270-4697-b13c-f7e4a1d92d7c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:44.497477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e34005a18a472b82a55ff84cced9c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '489dd2d53f174e62baa25f0b53b1a6cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:44.498388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:44.498744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:44.499029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] PATH_INFO: `/v3/projects/661d219445a4478e886f23eddf70dd54` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:44.500471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: Authorizing `identity:delete_project(project_id=661d219445a4478e886f23eddf70dd54)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:44.503888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:44.563897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:44.567022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '661d219445a4478e886f23eddf70dd54', 'request_id': 'req-48c96153-5fd6-4b6f-9509-fb48882a2b8c'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:44.577210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '661d219445a4478e886f23eddf70dd54', 'request_id': 'req-48c96153-5fd6-4b6f-9509-fb48882a2b8c'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:44.583475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '661d219445a4478e886f23eddf70dd54', 'request_id': 'req-48c96153-5fd6-4b6f-9509-fb48882a2b8c'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:44.595820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] The token cache is being invalidate because project 661d219445a4478e886f23eddf70dd54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:44.596117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:44.596810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-48c96153-5fd6-4b6f-9509-fb48882a2b8c tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:44.598421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 988/1982] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:44 2022] DELETE /identity/v3/projects/661d219445a4478e886f23eddf70dd54 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:44.804359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31d26d-1b12-4745-85eb-355a5f0e8f10 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:44.806124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3a31d26d-1b12-4745-85eb-355a5f0e8f10 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:44.888635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31d26d-1b12-4745-85eb-355a5f0e8f10 tempest-VolumeQuotaClassesTest-441405914 tempest-VolumeQuotaClassesTest-441405914-project-admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:44.932655 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:44.956127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:44.957018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:44.957296 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:44.957568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] PATH_INFO: `/v3/users/cbfed2248f8344b7959ee7ee41f2e3f5` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:44.958805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbfed2248f8344b7959ee7ee41f2e3f5)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:44.961283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:45.030731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbfed2248f8344b7959ee7ee41f2e3f5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.041122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbfed2248f8344b7959ee7ee41f2e3f5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.044544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbfed2248f8344b7959ee7ee41f2e3f5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.044544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbfed2248f8344b7959ee7ee41f2e3f5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.063757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbfed2248f8344b7959ee7ee41f2e3f5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.089847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 995/1983] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:44 2022] DELETE /identity/v3/users/cbfed2248f8344b7959ee7ee41f2e3f5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:45.094830 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96e2ed-e26c-4c4e-aa63-b024b826ceed None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:45.095366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96e2ed-e26c-4c4e-aa63-b024b826ceed None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:45.095539 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96e2ed-e26c-4c4e-aa63-b024b826ceed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:45.114466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-2f96e2ed-e26c-4c4e-aa63-b024b826ceed None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:45.115769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2f96e2ed-e26c-4c4e-aa63-b024b826ceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:45.198070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 989/1984] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:45.208234 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:45.209903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:45.317135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e6542-b2e9-43e0-98a3-a84c19419053 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:45.321368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e78bf-e47b-40a0-aa7b-8580bda61d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:45.322020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e78bf-e47b-40a0-aa7b-8580bda61d5e service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:45.322332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e78bf-e47b-40a0-aa7b-8580bda61d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:45.322613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e78bf-e47b-40a0-aa7b-8580bda61d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:45.331389 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e78bf-e47b-40a0-aa7b-8580bda61d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:45.332853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e78bf-e47b-40a0-aa7b-8580bda61d5e service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:45.342801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 996/1985] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:45.618829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96e2ed-e26c-4c4e-aa63-b024b826ceed None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:45.620294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2f96e2ed-e26c-4c4e-aa63-b024b826ceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:45.633075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96e2ed-e26c-4c4e-aa63-b024b826ceed None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:45.637882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:45.638795 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:45.639084 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:45.639362 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] PATH_INFO: `/v3/projects/665e8ebc26394f92befa75a61f90e792` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:45.640562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=665e8ebc26394f92befa75a61f90e792)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:45.642521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:45.688926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '665e8ebc26394f92befa75a61f90e792', 'request_id': 'req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.696419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '665e8ebc26394f92befa75a61f90e792', 'request_id': 'req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.701844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '665e8ebc26394f92befa75a61f90e792', 'request_id': 'req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.712610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] The token cache is being invalidate because project 665e8ebc26394f92befa75a61f90e792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:45.712897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.713523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.715113 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 990/1986] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:45 2022] DELETE /identity/v3/projects/665e8ebc26394f92befa75a61f90e792 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:45.719236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e78bf-e47b-40a0-aa7b-8580bda61d5e service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:45.728742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e78bf-e47b-40a0-aa7b-8580bda61d5e service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:45.787607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114526e6-b981-46cf-ad09-ed74f140fb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:45.788631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114526e6-b981-46cf-ad09-ed74f140fb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:45.788892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114526e6-b981-46cf-ad09-ed74f140fb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:45.789170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114526e6-b981-46cf-ad09-ed74f140fb70 admin admin] PATH_INFO: `/v3/users/19e34005a18a472b82a55ff84cced9c7` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:45.790541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114526e6-b981-46cf-ad09-ed74f140fb70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e34005a18a472b82a55ff84cced9c7)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:45.792489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114526e6-b981-46cf-ad09-ed74f140fb70 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:45.797606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:45.853696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-114526e6-b981-46cf-ad09-ed74f140fb70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e34005a18a472b82a55ff84cced9c7'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.863372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-114526e6-b981-46cf-ad09-ed74f140fb70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e34005a18a472b82a55ff84cced9c7'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.863979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-114526e6-b981-46cf-ad09-ed74f140fb70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e34005a18a472b82a55ff84cced9c7'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.864595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-114526e6-b981-46cf-ad09-ed74f140fb70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e34005a18a472b82a55ff84cced9c7'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.881620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-114526e6-b981-46cf-ad09-ed74f140fb70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e34005a18a472b82a55ff84cced9c7'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:45.894100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 997/1987] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:45 2022] DELETE /identity/v3/users/19e34005a18a472b82a55ff84cced9c7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:46.200110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:46.201832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:46.218272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e800bf-29a8-4f27-a3ce-3c0e400c79d3 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:46.329029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:46.352042 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:46.353420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:46.353420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:46.353624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] PATH_INFO: `/v3/projects/489dd2d53f174e62baa25f0b53b1a6cc` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:46.355465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=489dd2d53f174e62baa25f0b53b1a6cc)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:46.357563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:46.434112 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '489dd2d53f174e62baa25f0b53b1a6cc', 'request_id': 'req-ed77355c-0811-401d-9f97-0427b4f2e8d4'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:46.434750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '489dd2d53f174e62baa25f0b53b1a6cc', 'request_id': 'req-ed77355c-0811-401d-9f97-0427b4f2e8d4'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:46.441645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '489dd2d53f174e62baa25f0b53b1a6cc', 'request_id': 'req-ed77355c-0811-401d-9f97-0427b4f2e8d4'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:46.454530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] The token cache is being invalidate because project 489dd2d53f174e62baa25f0b53b1a6cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:46.454898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:46.455624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed77355c-0811-401d-9f97-0427b4f2e8d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:46.457494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 991/1988] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:46 2022] DELETE /identity/v3/projects/489dd2d53f174e62baa25f0b53b1a6cc => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:46.462624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084821e-a05e-491b-ae4e-f0262a983722 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:46.462914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084821e-a05e-491b-ae4e-f0262a983722 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:46.463194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084821e-a05e-491b-ae4e-f0262a983722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:46.483334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e084821e-a05e-491b-ae4e-f0262a983722 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:46.484541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e084821e-a05e-491b-ae4e-f0262a983722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:46.547125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 998/1989] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:46.554855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f732d-eab5-4dd6-82b1-c31f53d8184e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:46.555147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f732d-eab5-4dd6-82b1-c31f53d8184e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:46.555423 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f732d-eab5-4dd6-82b1-c31f53d8184e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:46.573768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-797f732d-eab5-4dd6-82b1-c31f53d8184e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:46.575079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-797f732d-eab5-4dd6-82b1-c31f53d8184e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:46.582935 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 992/1990] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:46.587659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084821e-a05e-491b-ae4e-f0262a983722 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:46.597008 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084821e-a05e-491b-ae4e-f0262a983722 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:46.601915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a0640-94b2-4cee-b48a-a87b38ec43ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:46.602711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a0640-94b2-4cee-b48a-a87b38ec43ee admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:46.603104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a0640-94b2-4cee-b48a-a87b38ec43ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:46.603410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a0640-94b2-4cee-b48a-a87b38ec43ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:46.604212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a0640-94b2-4cee-b48a-a87b38ec43ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:01:46.604768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a0640-94b2-4cee-b48a-a87b38ec43ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:46.606264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a0640-94b2-4cee-b48a-a87b38ec43ee admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:46.618771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 999/1991] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:01:46.623076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f732d-eab5-4dd6-82b1-c31f53d8184e None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:46.634966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f732d-eab5-4dd6-82b1-c31f53d8184e None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:46.643782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b8ef6-fe88-4c71-8044-9adf6976704f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:46.644479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b8ef6-fe88-4c71-8044-9adf6976704f admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:46.644770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b8ef6-fe88-4c71-8044-9adf6976704f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:46.645022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b8ef6-fe88-4c71-8044-9adf6976704f admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:46.645908 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b8ef6-fe88-4c71-8044-9adf6976704f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:46.648489 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b8ef6-fe88-4c71-8044-9adf6976704f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:46.682259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 993/1992] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:46 2022] POST /identity/v3/projects => generated 409 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:46.686626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a0640-94b2-4cee-b48a-a87b38ec43ee admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:46.697475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a0640-94b2-4cee-b48a-a87b38ec43ee admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:46.765578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa2c53-d472-4ca1-8513-40e2a320fb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:46.765578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa2c53-d472-4ca1-8513-40e2a320fb70 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:46.765578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa2c53-d472-4ca1-8513-40e2a320fb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:46.765578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa2c53-d472-4ca1-8513-40e2a320fb70 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:46.766403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa2c53-d472-4ca1-8513-40e2a320fb70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:46.766484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa2c53-d472-4ca1-8513-40e2a320fb70 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:46.784264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:46.794004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1000/1993] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:46 2022] POST /identity/v3/users => generated 305 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:46.796064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b8ef6-fe88-4c71-8044-9adf6976704f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:46.809789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b8ef6-fe88-4c71-8044-9adf6976704f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:46.814916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d60c08-aa34-4e24-ac0c-d0c79b578c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:46.816441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d60c08-aa34-4e24-ac0c-d0c79b578c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:46.816821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d60c08-aa34-4e24-ac0c-d0c79b578c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:46.817175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d60c08-aa34-4e24-ac0c-d0c79b578c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:46.817988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d60c08-aa34-4e24-ac0c-d0c79b578c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:46.819965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d60c08-aa34-4e24-ac0c-d0c79b578c5e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:46.833131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 994/1994] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:46.836796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa2c53-d472-4ca1-8513-40e2a320fb70 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:46.847346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa2c53-d472-4ca1-8513-40e2a320fb70 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:46.854193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab57007-4568-4ea9-b2c5-08220b865709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:46.854964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab57007-4568-4ea9-b2c5-08220b865709 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:46.855323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab57007-4568-4ea9-b2c5-08220b865709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:46.855665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab57007-4568-4ea9-b2c5-08220b865709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:46.856474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab57007-4568-4ea9-b2c5-08220b865709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:46.858466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab57007-4568-4ea9-b2c5-08220b865709 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:46.869885 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1001/1995] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:46.874638 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d60c08-aa34-4e24-ac0c-d0c79b578c5e admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:46.886528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d60c08-aa34-4e24-ac0c-d0c79b578c5e admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:46.891574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d905f9-23a5-4248-9a1e-984df4b1e2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:46.892467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d905f9-23a5-4248-9a1e-984df4b1e2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:46.892776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d905f9-23a5-4248-9a1e-984df4b1e2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:46.893047 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d905f9-23a5-4248-9a1e-984df4b1e2c4 admin admin] PATH_INFO: `/v3/projects/0449c6fa471a48f3bb01f11e32ad65b2/users/609cfc8b4f444735a4495f40f5800fbd/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:46.909086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d905f9-23a5-4248-9a1e-984df4b1e2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0449c6fa471a48f3bb01f11e32ad65b2, user_id=609cfc8b4f444735a4495f40f5800fbd, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:46.911709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d905f9-23a5-4248-9a1e-984df4b1e2c4 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:46.927320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 995/1996] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:46 2022] PUT /identity/v3/projects/0449c6fa471a48f3bb01f11e32ad65b2/users/609cfc8b4f444735a4495f40f5800fbd/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:01:46.939232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d76e6-c95d-4cb7-b8c2-83e5cee5d448 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:46.939531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d76e6-c95d-4cb7-b8c2-83e5cee5d448 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:46.939816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d76e6-c95d-4cb7-b8c2-83e5cee5d448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:46.970536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f26d76e6-c95d-4cb7-b8c2-83e5cee5d448 None None] MFA Rules not processed for user `609cfc8b4f444735a4495f40f5800fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:46.971913 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f26d76e6-c95d-4cb7-b8c2-83e5cee5d448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:47.038925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1002/1997] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:46 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:47.044826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d905f9-23a5-4248-9a1e-984df4b1e2c4 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:47.109107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d905f9-23a5-4248-9a1e-984df4b1e2c4 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:47.114214 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e92e8-c8ad-4cf0-b967-87a1deaf5b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:47.115016 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e92e8-c8ad-4cf0-b967-87a1deaf5b34 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:47.115310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e92e8-c8ad-4cf0-b967-87a1deaf5b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:47.115582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e92e8-c8ad-4cf0-b967-87a1deaf5b34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:47.116492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e92e8-c8ad-4cf0-b967-87a1deaf5b34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:47.118891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e92e8-c8ad-4cf0-b967-87a1deaf5b34 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:47.151656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 996/1998] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:47 2022] POST /identity/v3/projects => generated 409 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:47.152798 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d76e6-c95d-4cb7-b8c2-83e5cee5d448 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:47.165773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d76e6-c95d-4cb7-b8c2-83e5cee5d448 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:47.280985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5d7b8-6855-4b09-9aa1-d28f7f07a347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:47.282853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5d7b8-6855-4b09-9aa1-d28f7f07a347 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:47.283448 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5d7b8-6855-4b09-9aa1-d28f7f07a347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:47.283871 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5d7b8-6855-4b09-9aa1-d28f7f07a347 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:47.284962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5d7b8-6855-4b09-9aa1-d28f7f07a347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:47.287226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5d7b8-6855-4b09-9aa1-d28f7f07a347 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:47.310086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1003/1999] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:47 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:47.314434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e92e8-c8ad-4cf0-b967-87a1deaf5b34 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:47.324610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e92e8-c8ad-4cf0-b967-87a1deaf5b34 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:47.329380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4707bf5-0a08-4e41-aad3-7bddd6593c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:47.330094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4707bf5-0a08-4e41-aad3-7bddd6593c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:47.330421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4707bf5-0a08-4e41-aad3-7bddd6593c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:47.330698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4707bf5-0a08-4e41-aad3-7bddd6593c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:47.331407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4707bf5-0a08-4e41-aad3-7bddd6593c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:47.333289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4707bf5-0a08-4e41-aad3-7bddd6593c8e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:47.336788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:47.346622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 997/2000] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:47.352292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5d7b8-6855-4b09-9aa1-d28f7f07a347 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:47.362777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5d7b8-6855-4b09-9aa1-d28f7f07a347 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:47.368154 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c2e6f-db0f-4862-b06f-f37e30de6ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:47.369039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c2e6f-db0f-4862-b06f-f37e30de6ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:47.369326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c2e6f-db0f-4862-b06f-f37e30de6ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:47.369604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c2e6f-db0f-4862-b06f-f37e30de6ea9 admin admin] PATH_INFO: `/v3/projects/b2fe87669f9f422591beaa07cfb73f3a/users/ce55af9098984cafbf2ed024d77d8c3e/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:47.384686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c2e6f-db0f-4862-b06f-f37e30de6ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2fe87669f9f422591beaa07cfb73f3a, user_id=ce55af9098984cafbf2ed024d77d8c3e, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:47.387397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c2e6f-db0f-4862-b06f-f37e30de6ea9 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:47.409499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1004/2001] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:47 2022] PUT /identity/v3/projects/b2fe87669f9f422591beaa07cfb73f3a/users/ce55af9098984cafbf2ed024d77d8c3e/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:47.413786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4707bf5-0a08-4e41-aad3-7bddd6593c8e admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:47.424156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4707bf5-0a08-4e41-aad3-7bddd6593c8e admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:47.499293 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d32875-60ba-443f-91e0-3fd9d3e56250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:47.500303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d32875-60ba-443f-91e0-3fd9d3e56250 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:47.500303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d32875-60ba-443f-91e0-3fd9d3e56250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:47.500616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d32875-60ba-443f-91e0-3fd9d3e56250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:47.501645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d32875-60ba-443f-91e0-3fd9d3e56250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:47.511079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d32875-60ba-443f-91e0-3fd9d3e56250 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:47.526667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 998/2002] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:01:47.527987 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c2e6f-db0f-4862-b06f-f37e30de6ea9 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:47.539150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c2e6f-db0f-4862-b06f-f37e30de6ea9 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:47.551547 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83870192-aa87-4a5a-9c4b-ba5fa91150d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:47.552311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83870192-aa87-4a5a-9c4b-ba5fa91150d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:47.552592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83870192-aa87-4a5a-9c4b-ba5fa91150d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:47.552952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83870192-aa87-4a5a-9c4b-ba5fa91150d1 admin admin] PATH_INFO: `/v3/projects/b2fe87669f9f422591beaa07cfb73f3a/users/ce55af9098984cafbf2ed024d77d8c3e/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:47.555688 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83870192-aa87-4a5a-9c4b-ba5fa91150d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2fe87669f9f422591beaa07cfb73f3a, user_id=ce55af9098984cafbf2ed024d77d8c3e, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:47.558378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83870192-aa87-4a5a-9c4b-ba5fa91150d1 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:47.569205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1005/2003] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:47 2022] PUT /identity/v3/projects/b2fe87669f9f422591beaa07cfb73f3a/users/ce55af9098984cafbf2ed024d77d8c3e/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:47.580377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154106f0-9482-49e8-bea6-6f2f55a1442f None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:47.581309 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154106f0-9482-49e8-bea6-6f2f55a1442f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:47.581671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154106f0-9482-49e8-bea6-6f2f55a1442f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:47.616092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-154106f0-9482-49e8-bea6-6f2f55a1442f None None] MFA Rules not processed for user `ce55af9098984cafbf2ed024d77d8c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:47.618167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-154106f0-9482-49e8-bea6-6f2f55a1442f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:47.737393 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 999/2004] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:47 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:47.748629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83870192-aa87-4a5a-9c4b-ba5fa91150d1 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:47.750560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-83870192-aa87-4a5a-9c4b-ba5fa91150d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:47.812526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:47.858446 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83870192-aa87-4a5a-9c4b-ba5fa91150d1 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:47.863387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe682bc-98a4-4275-b607-ef1945e08afe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:47.865920 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe682bc-98a4-4275-b607-ef1945e08afe service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:47.866350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe682bc-98a4-4275-b607-ef1945e08afe service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:47.867231 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe682bc-98a4-4275-b607-ef1945e08afe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:47.880194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe682bc-98a4-4275-b607-ef1945e08afe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:47.882690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe682bc-98a4-4275-b607-ef1945e08afe service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:47.910346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1006/2005] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:47 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:48.147299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154106f0-9482-49e8-bea6-6f2f55a1442f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:48.158797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154106f0-9482-49e8-bea6-6f2f55a1442f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:48.164409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72f6ef-f54c-4ba9-b5d9-cc404c8c8c4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:48.165073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f6ef-f54c-4ba9-b5d9-cc404c8c8c4b service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:48.165356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f6ef-f54c-4ba9-b5d9-cc404c8c8c4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:48.165636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f6ef-f54c-4ba9-b5d9-cc404c8c8c4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:48.253785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72f6ef-f54c-4ba9-b5d9-cc404c8c8c4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:48.256293 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72f6ef-f54c-4ba9-b5d9-cc404c8c8c4b service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:48.269206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe682bc-98a4-4275-b607-ef1945e08afe service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:48.271140 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1000/2006] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:48 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:48.271535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-dfe682bc-98a4-4275-b607-ef1945e08afe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:48.346418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe682bc-98a4-4275-b607-ef1945e08afe service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:48.351415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:48.352282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:48.352635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:48.352980 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] PATH_INFO: `/v3/users/68c1ecd3d5f6421cabe0654ae61a866c` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:48.354354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c1ecd3d5f6421cabe0654ae61a866c)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:48.356290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:48.450803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c1ecd3d5f6421cabe0654ae61a866c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:48.467288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c1ecd3d5f6421cabe0654ae61a866c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:48.467967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c1ecd3d5f6421cabe0654ae61a866c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:48.469995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c1ecd3d5f6421cabe0654ae61a866c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:48.487496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c1ecd3d5f6421cabe0654ae61a866c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:48.506584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1007/2007] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:48 2022] DELETE /identity/v3/users/68c1ecd3d5f6421cabe0654ae61a866c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:48.512668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ec502-6264-4049-a919-ade8d592d7ba None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:48.513092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ec502-6264-4049-a919-ade8d592d7ba None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:48.513285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ec502-6264-4049-a919-ade8d592d7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:48.536841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a61ec502-6264-4049-a919-ade8d592d7ba None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:48.538249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a61ec502-6264-4049-a919-ade8d592d7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:48.558230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:48.637767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1001/2008] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:48.673450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:48.675528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:48.770719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303718b-949f-4b86-816b-d23b7c10ffbd admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:48.775220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb9866-6300-4615-a3cc-fcb9c6e37188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:48.775972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb9866-6300-4615-a3cc-fcb9c6e37188 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:48.776377 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb9866-6300-4615-a3cc-fcb9c6e37188 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:48.776749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb9866-6300-4615-a3cc-fcb9c6e37188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:48.785265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:48.789982 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb9866-6300-4615-a3cc-fcb9c6e37188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:48.792546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb9866-6300-4615-a3cc-fcb9c6e37188 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:48.809836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1008/2009] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:49.536181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ec502-6264-4049-a919-ade8d592d7ba None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:49.537786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a61ec502-6264-4049-a919-ade8d592d7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:49.554642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ec502-6264-4049-a919-ade8d592d7ba None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:49.560486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:49.561386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:49.561726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:49.562003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] PATH_INFO: `/v3/projects/f99fa8a74fdb4c5ab9ffe038301db2a9` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:49.563306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=f99fa8a74fdb4c5ab9ffe038301db2a9)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:49.565930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:49.638021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f99fa8a74fdb4c5ab9ffe038301db2a9', 'request_id': 'req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:49.648977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f99fa8a74fdb4c5ab9ffe038301db2a9', 'request_id': 'req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:49.657065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f99fa8a74fdb4c5ab9ffe038301db2a9', 'request_id': 'req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:49.673475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] The token cache is being invalidate because project f99fa8a74fdb4c5ab9ffe038301db2a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:49.673901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:49.675135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:49.677558 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1002/2010] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:49 2022] DELETE /identity/v3/projects/f99fa8a74fdb4c5ab9ffe038301db2a9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:49.680932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb9866-6300-4615-a3cc-fcb9c6e37188 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:49.691782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb9866-6300-4615-a3cc-fcb9c6e37188 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:49.759668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b607a75-09c9-4c35-8902-586d4325e4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:49.760503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b607a75-09c9-4c35-8902-586d4325e4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:49.760789 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b607a75-09c9-4c35-8902-586d4325e4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:49.761069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b607a75-09c9-4c35-8902-586d4325e4b5 admin admin] PATH_INFO: `/v3/users/5bc8a006ac1143d7b391ff32ddfaac21` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:49.762333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b607a75-09c9-4c35-8902-586d4325e4b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc8a006ac1143d7b391ff32ddfaac21)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:49.764440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b607a75-09c9-4c35-8902-586d4325e4b5 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:49.824908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2b607a75-09c9-4c35-8902-586d4325e4b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc8a006ac1143d7b391ff32ddfaac21'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:49.835474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2b607a75-09c9-4c35-8902-586d4325e4b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc8a006ac1143d7b391ff32ddfaac21'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:49.837658 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2b607a75-09c9-4c35-8902-586d4325e4b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc8a006ac1143d7b391ff32ddfaac21'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:49.838456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2b607a75-09c9-4c35-8902-586d4325e4b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc8a006ac1143d7b391ff32ddfaac21'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:49.853911 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2b607a75-09c9-4c35-8902-586d4325e4b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc8a006ac1143d7b391ff32ddfaac21'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:49.871465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1009/2011] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:49 2022] DELETE /identity/v3/users/5bc8a006ac1143d7b391ff32ddfaac21 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:50.279323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:50.281203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:50.359956 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ba997-88f6-4f9e-bdfd-c2b04b4e59ef admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:50.483931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:50.484870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:50.485161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:50.485440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] PATH_INFO: `/v3/projects/358cd22a40ab48f99d6f8f4d5d1009c4` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:50.487895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=358cd22a40ab48f99d6f8f4d5d1009c4)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:50.489855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:50.561032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:50.568021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '358cd22a40ab48f99d6f8f4d5d1009c4', 'request_id': 'req-4066e227-acee-4fc9-b7c5-6620cb091d6e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:50.583451 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '358cd22a40ab48f99d6f8f4d5d1009c4', 'request_id': 'req-4066e227-acee-4fc9-b7c5-6620cb091d6e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:50.590854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '358cd22a40ab48f99d6f8f4d5d1009c4', 'request_id': 'req-4066e227-acee-4fc9-b7c5-6620cb091d6e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:50.612447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] The token cache is being invalidate because project 358cd22a40ab48f99d6f8f4d5d1009c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:50.612748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:50.613424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4066e227-acee-4fc9-b7c5-6620cb091d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:50.615810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1003/2012] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:50 2022] DELETE /identity/v3/projects/358cd22a40ab48f99d6f8f4d5d1009c4 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:50.621967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc80b80-1a59-498f-9cfe-8c2397b90fe2 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:50.622255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc80b80-1a59-498f-9cfe-8c2397b90fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:50.622578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc80b80-1a59-498f-9cfe-8c2397b90fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:50.644521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-5bc80b80-1a59-498f-9cfe-8c2397b90fe2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:50.645901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5bc80b80-1a59-498f-9cfe-8c2397b90fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:50.761630 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1010/2013] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:50.764018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32e6d-d5a4-49e2-b71b-007655475a83 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:50.764305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32e6d-d5a4-49e2-b71b-007655475a83 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:50.764539 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32e6d-d5a4-49e2-b71b-007655475a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:50.803263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-c4b32e6d-d5a4-49e2-b71b-007655475a83 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:50.804746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c4b32e6d-d5a4-49e2-b71b-007655475a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:50.815511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1004/2014] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:01:50.821000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc80b80-1a59-498f-9cfe-8c2397b90fe2 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:50.834449 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc80b80-1a59-498f-9cfe-8c2397b90fe2 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:50.841199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c6a31-aad9-4266-ae66-204208934925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:50.842148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c6a31-aad9-4266-ae66-204208934925 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:50.842649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c6a31-aad9-4266-ae66-204208934925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:50.843143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c6a31-aad9-4266-ae66-204208934925 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:50.844103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c6a31-aad9-4266-ae66-204208934925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:01:50.844659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c6a31-aad9-4266-ae66-204208934925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:50.846611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c6a31-aad9-4266-ae66-204208934925 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:50.868360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b32e6d-d5a4-49e2-b71b-007655475a83 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:50.875805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1011/2015] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:01:50.902274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b32e6d-d5a4-49e2-b71b-007655475a83 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:50.918185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5634d6-eb8f-42bb-9cfc-fc54792b5d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:50.920383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5634d6-eb8f-42bb-9cfc-fc54792b5d21 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:50.920383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5634d6-eb8f-42bb-9cfc-fc54792b5d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:50.920734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5634d6-eb8f-42bb-9cfc-fc54792b5d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:50.921552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5634d6-eb8f-42bb-9cfc-fc54792b5d21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:50.924009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5634d6-eb8f-42bb-9cfc-fc54792b5d21 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:50.960853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1005/2016] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:50 2022] POST /identity/v3/projects => generated 389 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:50.962162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c6a31-aad9-4266-ae66-204208934925 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:50.967681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:50.973750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c6a31-aad9-4266-ae66-204208934925 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:51.046010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe6537-8360-4a23-aacc-5ab2296e4ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:51.046731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe6537-8360-4a23-aacc-5ab2296e4ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.047021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe6537-8360-4a23-aacc-5ab2296e4ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.047300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe6537-8360-4a23-aacc-5ab2296e4ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.048203 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe6537-8360-4a23-aacc-5ab2296e4ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:51.050147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe6537-8360-4a23-aacc-5ab2296e4ee6 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:51.074228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1012/2017] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:50 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:51.076048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5634d6-eb8f-42bb-9cfc-fc54792b5d21 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:51.086898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5634d6-eb8f-42bb-9cfc-fc54792b5d21 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:51.091607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12029e-ce27-4b8b-8fad-8edab25c47a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:51.092269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12029e-ce27-4b8b-8fad-8edab25c47a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.092608 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12029e-ce27-4b8b-8fad-8edab25c47a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.092893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12029e-ce27-4b8b-8fad-8edab25c47a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.093619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12029e-ce27-4b8b-8fad-8edab25c47a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:51.095252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12029e-ce27-4b8b-8fad-8edab25c47a9 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:51.106754 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1006/2018] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:51.110849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe6537-8360-4a23-aacc-5ab2296e4ee6 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:51.121001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe6537-8360-4a23-aacc-5ab2296e4ee6 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:51.125935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b484c36-ef52-4066-a4da-d3c9a1df74e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:51.126652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b484c36-ef52-4066-a4da-d3c9a1df74e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.126944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b484c36-ef52-4066-a4da-d3c9a1df74e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.127232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b484c36-ef52-4066-a4da-d3c9a1df74e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.127959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b484c36-ef52-4066-a4da-d3c9a1df74e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:51.129560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b484c36-ef52-4066-a4da-d3c9a1df74e6 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:51.139864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1013/2019] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:51.143761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12029e-ce27-4b8b-8fad-8edab25c47a9 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:51.154119 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12029e-ce27-4b8b-8fad-8edab25c47a9 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:51.158758 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743ecb1-a024-48d7-a231-7caab76b8c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:51.159636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743ecb1-a024-48d7-a231-7caab76b8c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.159921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743ecb1-a024-48d7-a231-7caab76b8c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.160199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743ecb1-a024-48d7-a231-7caab76b8c0c admin admin] PATH_INFO: `/v3/projects/57a789ca0f4a49a28929dbe935317da1/users/2adfb8d2720640ec9f4e7d57bb551f04/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.174535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743ecb1-a024-48d7-a231-7caab76b8c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a789ca0f4a49a28929dbe935317da1, user_id=2adfb8d2720640ec9f4e7d57bb551f04, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:51.177135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743ecb1-a024-48d7-a231-7caab76b8c0c admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:51.190520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1007/2020] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:51 2022] PUT /identity/v3/projects/57a789ca0f4a49a28929dbe935317da1/users/2adfb8d2720640ec9f4e7d57bb551f04/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:51.201156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d01bd1-8298-41f8-b44d-94b55f434f45 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.201455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d01bd1-8298-41f8-b44d-94b55f434f45 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.201744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d01bd1-8298-41f8-b44d-94b55f434f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.228220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-58d01bd1-8298-41f8-b44d-94b55f434f45 None None] MFA Rules not processed for user `2adfb8d2720640ec9f4e7d57bb551f04`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:51.229425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-58d01bd1-8298-41f8-b44d-94b55f434f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:51.292881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1014/2021] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:51.297488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743ecb1-a024-48d7-a231-7caab76b8c0c admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:51.357673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743ecb1-a024-48d7-a231-7caab76b8c0c admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:51.362256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedec6f9-1bd2-4647-9857-83c0b9bb74eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:51.362941 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedec6f9-1bd2-4647-9857-83c0b9bb74eb admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.363227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedec6f9-1bd2-4647-9857-83c0b9bb74eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.363500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedec6f9-1bd2-4647-9857-83c0b9bb74eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.364208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedec6f9-1bd2-4647-9857-83c0b9bb74eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:51.366147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedec6f9-1bd2-4647-9857-83c0b9bb74eb admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:51.391666 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1008/2022] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:01:51 2022] POST /identity/v3/projects => generated 391 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:51.395622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d01bd1-8298-41f8-b44d-94b55f434f45 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:51.406988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d01bd1-8298-41f8-b44d-94b55f434f45 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:51.465249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994f11f-93f8-465d-97fe-9986dc65970c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:51.465843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f11f-93f8-465d-97fe-9986dc65970c admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.466124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f11f-93f8-465d-97fe-9986dc65970c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.466443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f11f-93f8-465d-97fe-9986dc65970c admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.467138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994f11f-93f8-465d-97fe-9986dc65970c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:51.469085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994f11f-93f8-465d-97fe-9986dc65970c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:51.484080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:51.489705 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1015/2023] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:01:51 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:01:51.494124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedec6f9-1bd2-4647-9857-83c0b9bb74eb admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:51.504540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedec6f9-1bd2-4647-9857-83c0b9bb74eb admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:51.510001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e234e90-0398-4b4a-900c-f1c32277d409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:51.510711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e234e90-0398-4b4a-900c-f1c32277d409 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.511030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e234e90-0398-4b4a-900c-f1c32277d409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.511404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e234e90-0398-4b4a-900c-f1c32277d409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.512212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e234e90-0398-4b4a-900c-f1c32277d409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:51.513811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e234e90-0398-4b4a-900c-f1c32277d409 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:51.524806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1009/2024] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:51.528892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994f11f-93f8-465d-97fe-9986dc65970c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:51.539257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994f11f-93f8-465d-97fe-9986dc65970c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:51.543879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee3de1-c34d-44c2-af3d-24fd24d26d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:51.544750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee3de1-c34d-44c2-af3d-24fd24d26d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.545031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee3de1-c34d-44c2-af3d-24fd24d26d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.545305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee3de1-c34d-44c2-af3d-24fd24d26d29 admin admin] PATH_INFO: `/v3/projects/e1d69fb77e324e63b5d7199e7b6c95e0/users/4809ddcee4b34a7dbe6b1b17a4ed438f/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.564638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee3de1-c34d-44c2-af3d-24fd24d26d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d69fb77e324e63b5d7199e7b6c95e0, user_id=4809ddcee4b34a7dbe6b1b17a4ed438f, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:51.567417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee3de1-c34d-44c2-af3d-24fd24d26d29 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:51.585569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1016/2025] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:51 2022] PUT /identity/v3/projects/e1d69fb77e324e63b5d7199e7b6c95e0/users/4809ddcee4b34a7dbe6b1b17a4ed438f/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:51.589513 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e234e90-0398-4b4a-900c-f1c32277d409 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:51.600162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e234e90-0398-4b4a-900c-f1c32277d409 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:51.640452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:51.659295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643787cf-a52b-4191-a87e-5033a558d47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:51.659883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643787cf-a52b-4191-a87e-5033a558d47f admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.660172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643787cf-a52b-4191-a87e-5033a558d47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.660507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643787cf-a52b-4191-a87e-5033a558d47f admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.661167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643787cf-a52b-4191-a87e-5033a558d47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:51.662812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643787cf-a52b-4191-a87e-5033a558d47f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:51.674866 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1010/2026] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:01:51.679599 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee3de1-c34d-44c2-af3d-24fd24d26d29 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:51.689933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee3de1-c34d-44c2-af3d-24fd24d26d29 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:51.695236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053ee82-33e2-462b-a406-8f944fab52bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:51.696256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053ee82-33e2-462b-a406-8f944fab52bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.696584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053ee82-33e2-462b-a406-8f944fab52bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.696905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053ee82-33e2-462b-a406-8f944fab52bf admin admin] PATH_INFO: `/v3/projects/e1d69fb77e324e63b5d7199e7b6c95e0/users/4809ddcee4b34a7dbe6b1b17a4ed438f/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.698933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053ee82-33e2-462b-a406-8f944fab52bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d69fb77e324e63b5d7199e7b6c95e0, user_id=4809ddcee4b34a7dbe6b1b17a4ed438f, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:51.701589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053ee82-33e2-462b-a406-8f944fab52bf admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:51.712664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1017/2027] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:01:51 2022] PUT /identity/v3/projects/e1d69fb77e324e63b5d7199e7b6c95e0/users/4809ddcee4b34a7dbe6b1b17a4ed438f/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:51.723309 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a6249-1f87-4fdd-8079-d749b8bbd254 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.723529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a6249-1f87-4fdd-8079-d749b8bbd254 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.723810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a6249-1f87-4fdd-8079-d749b8bbd254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.751408 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9a0a6249-1f87-4fdd-8079-d749b8bbd254 None None] MFA Rules not processed for user `4809ddcee4b34a7dbe6b1b17a4ed438f`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:51.752661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9a0a6249-1f87-4fdd-8079-d749b8bbd254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:51.832814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1011/2028] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:51.843385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053ee82-33e2-462b-a406-8f944fab52bf admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:51.845592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e053ee82-33e2-462b-a406-8f944fab52bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:51.907515 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053ee82-33e2-462b-a406-8f944fab52bf admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:51.912078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b34abe-ac25-403b-8f6e-321bdd9ce484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:51.912739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b34abe-ac25-403b-8f6e-321bdd9ce484 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:51.913024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b34abe-ac25-403b-8f6e-321bdd9ce484 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:51.914079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b34abe-ac25-403b-8f6e-321bdd9ce484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:51.927301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b34abe-ac25-403b-8f6e-321bdd9ce484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:51.929343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b34abe-ac25-403b-8f6e-321bdd9ce484 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:51.939326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1018/2029] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:51.987335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a6249-1f87-4fdd-8079-d749b8bbd254 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:51.998529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a6249-1f87-4fdd-8079-d749b8bbd254 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:52.003988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9faa4-ad28-470b-97f6-f9df84861133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:52.005385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9faa4-ad28-470b-97f6-f9df84861133 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:52.005890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9faa4-ad28-470b-97f6-f9df84861133 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:52.006444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9faa4-ad28-470b-97f6-f9df84861133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:52.075433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9faa4-ad28-470b-97f6-f9df84861133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:52.076859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9faa4-ad28-470b-97f6-f9df84861133 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:52.086199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1012/2030] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:58.136256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b34abe-ac25-403b-8f6e-321bdd9ce484 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:58.138421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-54b34abe-ac25-403b-8f6e-321bdd9ce484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:58.236558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b34abe-ac25-403b-8f6e-321bdd9ce484 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:58.244082 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:58.245358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:58.246008 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:58.246432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] PATH_INFO: `/v3/users/609cfc8b4f444735a4495f40f5800fbd` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:58.250457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=609cfc8b4f444735a4495f40f5800fbd)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:58.251081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:58.311612 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '609cfc8b4f444735a4495f40f5800fbd'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:58.321206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '609cfc8b4f444735a4495f40f5800fbd'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:58.322449 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '609cfc8b4f444735a4495f40f5800fbd'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:58.323896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '609cfc8b4f444735a4495f40f5800fbd'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:58.337898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:58.340616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '609cfc8b4f444735a4495f40f5800fbd'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:58.363407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1019/2031] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:58 2022] DELETE /identity/v3/users/609cfc8b4f444735a4495f40f5800fbd => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:58.368490 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a871a7-ff9c-48f3-88cf-4155b1734dfc None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:58.369088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a871a7-ff9c-48f3-88cf-4155b1734dfc None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:58.369278 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a871a7-ff9c-48f3-88cf-4155b1734dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:58.387826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-f0a871a7-ff9c-48f3-88cf-4155b1734dfc None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:58.389054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f0a871a7-ff9c-48f3-88cf-4155b1734dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:58.471123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1013/2032] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:01:58.480690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:58.482236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:58.544794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf6853-9548-41e9-949c-d5dcaf2d1cb5 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:58.548591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f2eff-cf08-46f2-be26-02e365b95b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:58.549254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f2eff-cf08-46f2-be26-02e365b95b5d service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:58.549537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f2eff-cf08-46f2-be26-02e365b95b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:58.549811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f2eff-cf08-46f2-be26-02e365b95b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:58.561793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f2eff-cf08-46f2-be26-02e365b95b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:58.563355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f2eff-cf08-46f2-be26-02e365b95b5d service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:58.574819 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1020/2033] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:01:58.884029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a871a7-ff9c-48f3-88cf-4155b1734dfc None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:58.885570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f0a871a7-ff9c-48f3-88cf-4155b1734dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:58.892910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:58.901628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a871a7-ff9c-48f3-88cf-4155b1734dfc None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:58.907459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:58.909345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:58.909768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:58.910184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] PATH_INFO: `/v3/projects/0449c6fa471a48f3bb01f11e32ad65b2` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:58.911481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0449c6fa471a48f3bb01f11e32ad65b2)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:58.913882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:58.937456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f2eff-cf08-46f2-be26-02e365b95b5d service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:58.938909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f81f2eff-cf08-46f2-be26-02e365b95b5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:58.954980 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f2eff-cf08-46f2-be26-02e365b95b5d service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:58.962814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035246e-6aaf-43a2-8b6b-54fe485451a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:58.964205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035246e-6aaf-43a2-8b6b-54fe485451a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:58.964583 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035246e-6aaf-43a2-8b6b-54fe485451a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:58.964971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035246e-6aaf-43a2-8b6b-54fe485451a4 admin admin] PATH_INFO: `/v3/users/2adfb8d2720640ec9f4e7d57bb551f04` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:58.966365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035246e-6aaf-43a2-8b6b-54fe485451a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2adfb8d2720640ec9f4e7d57bb551f04)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:58.969456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035246e-6aaf-43a2-8b6b-54fe485451a4 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:58.981574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0449c6fa471a48f3bb01f11e32ad65b2', 'request_id': 'req-e6a5c491-f3ed-4dcf-a128-05055c3de93e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:58.989387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0449c6fa471a48f3bb01f11e32ad65b2', 'request_id': 'req-e6a5c491-f3ed-4dcf-a128-05055c3de93e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:58.999540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0449c6fa471a48f3bb01f11e32ad65b2', 'request_id': 'req-e6a5c491-f3ed-4dcf-a128-05055c3de93e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.017841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] The token cache is being invalidate because project 0449c6fa471a48f3bb01f11e32ad65b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:01:59.018077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.018792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.020972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1014/2034] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:58 2022] DELETE /identity/v3/projects/0449c6fa471a48f3bb01f11e32ad65b2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:01:59.025492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:59.026869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:59.035662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5035246e-6aaf-43a2-8b6b-54fe485451a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2adfb8d2720640ec9f4e7d57bb551f04'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.048124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5035246e-6aaf-43a2-8b6b-54fe485451a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2adfb8d2720640ec9f4e7d57bb551f04'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.048903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5035246e-6aaf-43a2-8b6b-54fe485451a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2adfb8d2720640ec9f4e7d57bb551f04'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.049743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5035246e-6aaf-43a2-8b6b-54fe485451a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2adfb8d2720640ec9f4e7d57bb551f04'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.068942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5035246e-6aaf-43a2-8b6b-54fe485451a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2adfb8d2720640ec9f4e7d57bb551f04'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.088855 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1021/2035] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:58 2022] DELETE /identity/v3/users/2adfb8d2720640ec9f4e7d57bb551f04 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:01:59.095062 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff919f59-4122-4657-9e0f-e4b81679dcf7 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:59.095501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff919f59-4122-4657-9e0f-e4b81679dcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:59.095930 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff919f59-4122-4657-9e0f-e4b81679dcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:59.103392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5c491-f3ed-4dcf-a128-05055c3de93e admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:59.121076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ff919f59-4122-4657-9e0f-e4b81679dcf7 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:01:59.122304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ff919f59-4122-4657-9e0f-e4b81679dcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:59.170570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:59.172557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:59.172967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:59.173357 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] PATH_INFO: `/v3/users/ce55af9098984cafbf2ed024d77d8c3e` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:59.174997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce55af9098984cafbf2ed024d77d8c3e)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:59.176955 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:59.181386 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:59.192592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1022/2036] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:01:59.204372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff919f59-4122-4657-9e0f-e4b81679dcf7 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:59.206410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ff919f59-4122-4657-9e0f-e4b81679dcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:59.219662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:01:59.232260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce55af9098984cafbf2ed024d77d8c3e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.246030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce55af9098984cafbf2ed024d77d8c3e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.255278 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce55af9098984cafbf2ed024d77d8c3e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.256110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce55af9098984cafbf2ed024d77d8c3e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.266004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce55af9098984cafbf2ed024d77d8c3e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:01:59.266700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff919f59-4122-4657-9e0f-e4b81679dcf7 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:59.270514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ec6e2-96d6-4667-a8b2-f16f31461839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:59.271308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ec6e2-96d6-4667-a8b2-f16f31461839 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:59.271590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ec6e2-96d6-4667-a8b2-f16f31461839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:59.271861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ec6e2-96d6-4667-a8b2-f16f31461839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:59.272719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1015/2037] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:01:59 2022] DELETE /identity/v3/users/ce55af9098984cafbf2ed024d77d8c3e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:01:59.273679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d21ec6e2-96d6-4667-a8b2-f16f31461839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:59.285949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ec6e2-96d6-4667-a8b2-f16f31461839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:59.287662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ec6e2-96d6-4667-a8b2-f16f31461839 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:59.299027 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1023/2038] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:01:59.836522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:59.839919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:59.845959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ec6e2-96d6-4667-a8b2-f16f31461839 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:01:59.847929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d21ec6e2-96d6-4667-a8b2-f16f31461839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:01:59.893100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ec6e2-96d6-4667-a8b2-f16f31461839 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:59.921812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a888430-2343-48cd-b8d2-957251023621 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:01:59.928000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:59.928843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:59.929130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:59.929407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] PATH_INFO: `/v3/projects/b2fe87669f9f422591beaa07cfb73f3a` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:59.930668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2fe87669f9f422591beaa07cfb73f3a)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:59.933068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:59.972268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:01:59.973236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:01:59.973519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:01:59.973793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] PATH_INFO: `/v3/projects/57a789ca0f4a49a28929dbe935317da1` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:01:59.975014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a789ca0f4a49a28929dbe935317da1)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:01:59.977534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:01:59.998504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2fe87669f9f422591beaa07cfb73f3a', 'request_id': 'req-fa922096-dc75-4c37-b399-f33c947db874'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.007425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2fe87669f9f422591beaa07cfb73f3a', 'request_id': 'req-fa922096-dc75-4c37-b399-f33c947db874'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.013847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2fe87669f9f422591beaa07cfb73f3a', 'request_id': 'req-fa922096-dc75-4c37-b399-f33c947db874'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.027611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] The token cache is being invalidate because project b2fe87669f9f422591beaa07cfb73f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:00.027854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.028518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fa922096-dc75-4c37-b399-f33c947db874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.033348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1016/2039] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:59 2022] DELETE /identity/v3/projects/b2fe87669f9f422591beaa07cfb73f3a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:00.035120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ac287-8221-4aa2-abf1-fe8a55a4dacf None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.035244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ac287-8221-4aa2-abf1-fe8a55a4dacf None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.035564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ac287-8221-4aa2-abf1-fe8a55a4dacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.051873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a789ca0f4a49a28929dbe935317da1', 'request_id': 'req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.055639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9a3ac287-8221-4aa2-abf1-fe8a55a4dacf None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:00.057142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9a3ac287-8221-4aa2-abf1-fe8a55a4dacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:00.061465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a789ca0f4a49a28929dbe935317da1', 'request_id': 'req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.067663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a789ca0f4a49a28929dbe935317da1', 'request_id': 'req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.079320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] The token cache is being invalidate because project 57a789ca0f4a49a28929dbe935317da1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:00.079640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.080292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.081858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1024/2040] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:01:59 2022] DELETE /identity/v3/projects/57a789ca0f4a49a28929dbe935317da1 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:00.086397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.087714 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:00.103486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:00.125753 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:00.128485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1017/2041] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:00.136919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f693-ae5d-470b-9e57-75c00c898204 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.137363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f693-ae5d-470b-9e57-75c00c898204 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.137662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f693-ae5d-470b-9e57-75c00c898204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.156664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-0f08f693-ae5d-470b-9e57-75c00c898204 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:00.158005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-0f08f693-ae5d-470b-9e57-75c00c898204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:00.159517 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda3fa9-92c7-41cd-b643-e4fb3c76fcef admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:00.164436 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:00.165311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.165592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.165915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] PATH_INFO: `/v3/users/4809ddcee4b34a7dbe6b1b17a4ed438f` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.166148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1018/2042] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:00.167167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4809ddcee4b34a7dbe6b1b17a4ed438f)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:00.169158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:00.170164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08f693-ae5d-470b-9e57-75c00c898204 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.179105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08f693-ae5d-470b-9e57-75c00c898204 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:00.184202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c74749-492b-49ce-8e9e-9c1e8042dd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:00.184938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c74749-492b-49ce-8e9e-9c1e8042dd0f admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.185311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c74749-492b-49ce-8e9e-9c1e8042dd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.185588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c74749-492b-49ce-8e9e-9c1e8042dd0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.186372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c74749-492b-49ce-8e9e-9c1e8042dd0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:02:00.186956 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c74749-492b-49ce-8e9e-9c1e8042dd0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:00.188600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c74749-492b-49ce-8e9e-9c1e8042dd0f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:00.205215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1019/2043] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:02:00.206375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c74749-492b-49ce-8e9e-9c1e8042dd0f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.216784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c74749-492b-49ce-8e9e-9c1e8042dd0f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:00.223466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dacfad-cb4a-4fbe-a0cf-5291c8a85f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:00.224345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dacfad-cb4a-4fbe-a0cf-5291c8a85f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.224546 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dacfad-cb4a-4fbe-a0cf-5291c8a85f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.224825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dacfad-cb4a-4fbe-a0cf-5291c8a85f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.225136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4809ddcee4b34a7dbe6b1b17a4ed438f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.225713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dacfad-cb4a-4fbe-a0cf-5291c8a85f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:00.227760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dacfad-cb4a-4fbe-a0cf-5291c8a85f0a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:00.232581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4809ddcee4b34a7dbe6b1b17a4ed438f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.233360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4809ddcee4b34a7dbe6b1b17a4ed438f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.234045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4809ddcee4b34a7dbe6b1b17a4ed438f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.248380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4809ddcee4b34a7dbe6b1b17a4ed438f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.257067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1020/2044] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:00 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:02:00.260704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dacfad-cb4a-4fbe-a0cf-5291c8a85f0a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.262608 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-73dacfad-cb4a-4fbe-a0cf-5291c8a85f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:00.265194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1025/2045] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:00 2022] DELETE /identity/v3/users/4809ddcee4b34a7dbe6b1b17a4ed438f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:00.332933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dacfad-cb4a-4fbe-a0cf-5291c8a85f0a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:00.401276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab8ec7-cae7-4a4b-9b09-28135f10cb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:00.402133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab8ec7-cae7-4a4b-9b09-28135f10cb5a admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.402458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab8ec7-cae7-4a4b-9b09-28135f10cb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.402738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab8ec7-cae7-4a4b-9b09-28135f10cb5a admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.403454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab8ec7-cae7-4a4b-9b09-28135f10cb5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:00.405608 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab8ec7-cae7-4a4b-9b09-28135f10cb5a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:00.431632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1021/2046] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:00 2022] POST /identity/v3/users => generated 281 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:02:00.435908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.447015 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df799d-16cc-437c-b4c2-1e45f8ed854d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:00.452772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53eddb-318e-404a-a789-83a57f865d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:00.454048 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53eddb-318e-404a-a789-83a57f865d11 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.454048 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53eddb-318e-404a-a789-83a57f865d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.454245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53eddb-318e-404a-a789-83a57f865d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.455041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53eddb-318e-404a-a789-83a57f865d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:00.456727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53eddb-318e-404a-a789-83a57f865d11 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:00.470907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1026/2047] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:00.475272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab8ec7-cae7-4a4b-9b09-28135f10cb5a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.489018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab8ec7-cae7-4a4b-9b09-28135f10cb5a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:00.495092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641599a4-c48c-48eb-a449-58e084c59d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:00.496033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641599a4-c48c-48eb-a449-58e084c59d95 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.496323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641599a4-c48c-48eb-a449-58e084c59d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.496626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641599a4-c48c-48eb-a449-58e084c59d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.497361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641599a4-c48c-48eb-a449-58e084c59d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:00.499025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641599a4-c48c-48eb-a449-58e084c59d95 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:00.513586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1022/2048] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:00.517877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53eddb-318e-404a-a789-83a57f865d11 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.529100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53eddb-318e-404a-a789-83a57f865d11 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:00.535793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1833257-3ac8-4d04-bd72-759104667e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:00.536683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1833257-3ac8-4d04-bd72-759104667e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.536968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1833257-3ac8-4d04-bd72-759104667e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.537245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1833257-3ac8-4d04-bd72-759104667e79 admin admin] PATH_INFO: `/v3/projects/db45471d682b456396cd7b5c055e0a0c/users/6492ce80a6b24ea994d9b1e24bc43eed/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.554520 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1833257-3ac8-4d04-bd72-759104667e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db45471d682b456396cd7b5c055e0a0c, user_id=6492ce80a6b24ea994d9b1e24bc43eed, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:00.557414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1833257-3ac8-4d04-bd72-759104667e79 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:00.572404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641599a4-c48c-48eb-a449-58e084c59d95 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.573570 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1027/2049] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:00 2022] PUT /identity/v3/projects/db45471d682b456396cd7b5c055e0a0c/users/6492ce80a6b24ea994d9b1e24bc43eed/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:00.574703 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-641599a4-c48c-48eb-a449-58e084c59d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:00.585804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310c903-0089-4df3-bf39-9ef163db140f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.586109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310c903-0089-4df3-bf39-9ef163db140f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.586392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310c903-0089-4df3-bf39-9ef163db140f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.593778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641599a4-c48c-48eb-a449-58e084c59d95 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:00.629572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f310c903-0089-4df3-bf39-9ef163db140f None None] MFA Rules not processed for user `6492ce80a6b24ea994d9b1e24bc43eed`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:00.630888 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f310c903-0089-4df3-bf39-9ef163db140f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:00.632640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:00.677354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:00.677592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.677854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.678078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] PATH_INFO: `/v3/projects/e1d69fb77e324e63b5d7199e7b6c95e0` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.679620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1d69fb77e324e63b5d7199e7b6c95e0)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:00.681536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:00.714025 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1028/2050] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:02:00.749406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1d69fb77e324e63b5d7199e7b6c95e0', 'request_id': 'req-0b5df5bb-beac-4ac2-8683-af934fa3f70f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.751873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310c903-0089-4df3-bf39-9ef163db140f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.753553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f310c903-0089-4df3-bf39-9ef163db140f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:00.758133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1d69fb77e324e63b5d7199e7b6c95e0', 'request_id': 'req-0b5df5bb-beac-4ac2-8683-af934fa3f70f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.765258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:00.765651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1d69fb77e324e63b5d7199e7b6c95e0', 'request_id': 'req-0b5df5bb-beac-4ac2-8683-af934fa3f70f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.778335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] The token cache is being invalidate because project e1d69fb77e324e63b5d7199e7b6c95e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:00.778627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.779297 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b5df5bb-beac-4ac2-8683-af934fa3f70f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:00.781066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1023/2051] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:00 2022] DELETE /identity/v3/projects/e1d69fb77e324e63b5d7199e7b6c95e0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:00.787364 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b1a37-187a-4bd6-a9ac-68439044d8f8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.788217 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b1a37-187a-4bd6-a9ac-68439044d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.788647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b1a37-187a-4bd6-a9ac-68439044d8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.808038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-074b1a37-187a-4bd6-a9ac-68439044d8f8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:00.809244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-074b1a37-187a-4bd6-a9ac-68439044d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:00.834414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310c903-0089-4df3-bf39-9ef163db140f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:00.876347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1024/2052] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:02:00.886229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d258-82f8-445a-8d7e-417e2d3e5cb5 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.886565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d258-82f8-445a-8d7e-417e2d3e5cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.886844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d258-82f8-445a-8d7e-417e2d3e5cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.888031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cd749-28af-4cbf-a6d9-81d42ed38375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:00.888767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cd749-28af-4cbf-a6d9-81d42ed38375 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.889058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cd749-28af-4cbf-a6d9-81d42ed38375 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.889352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cd749-28af-4cbf-a6d9-81d42ed38375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.891182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-735cd749-28af-4cbf-a6d9-81d42ed38375 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:00.906171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-3a51d258-82f8-445a-8d7e-417e2d3e5cb5 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:00.908244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3a51d258-82f8-445a-8d7e-417e2d3e5cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:00.918360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1025/2053] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:00.923239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51d258-82f8-445a-8d7e-417e2d3e5cb5 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.932733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51d258-82f8-445a-8d7e-417e2d3e5cb5 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:00.938593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59028a1b-e43f-4353-83f5-ea38c1e67baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:00.939486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59028a1b-e43f-4353-83f5-ea38c1e67baa admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.939778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59028a1b-e43f-4353-83f5-ea38c1e67baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.940055 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59028a1b-e43f-4353-83f5-ea38c1e67baa admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.940783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59028a1b-e43f-4353-83f5-ea38c1e67baa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:02:00.941594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59028a1b-e43f-4353-83f5-ea38c1e67baa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:00.944864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59028a1b-e43f-4353-83f5-ea38c1e67baa admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:00.957801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1026/2054] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:02:00.963132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59028a1b-e43f-4353-83f5-ea38c1e67baa admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.966104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cd749-28af-4cbf-a6d9-81d42ed38375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:00.967832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cd749-28af-4cbf-a6d9-81d42ed38375 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:00.975879 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59028a1b-e43f-4353-83f5-ea38c1e67baa admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:00.979698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1029/2055] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:02:00.980571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982efedd-7a23-4b2a-b9f4-05e61715258f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:00.981243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982efedd-7a23-4b2a-b9f4-05e61715258f admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:00.981531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982efedd-7a23-4b2a-b9f4-05e61715258f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:00.981815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982efedd-7a23-4b2a-b9f4-05e61715258f admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:00.982552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982efedd-7a23-4b2a-b9f4-05e61715258f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:00.984500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982efedd-7a23-4b2a-b9f4-05e61715258f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:00.996131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cd749-28af-4cbf-a6d9-81d42ed38375 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:00.998509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-735cd749-28af-4cbf-a6d9-81d42ed38375 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:01.023702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cd749-28af-4cbf-a6d9-81d42ed38375 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:01.032844 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7af22b-f0f2-4bbc-88eb-860b0a3d4caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:01.034120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7af22b-f0f2-4bbc-88eb-860b0a3d4caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.034686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7af22b-f0f2-4bbc-88eb-860b0a3d4caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.035098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7af22b-f0f2-4bbc-88eb-860b0a3d4caa admin admin] PATH_INFO: `/v3/users/6492ce80a6b24ea994d9b1e24bc43eed` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.038464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1027/2056] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:00 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:02:01.040882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982efedd-7a23-4b2a-b9f4-05e61715258f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:01.041460 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7af22b-f0f2-4bbc-88eb-860b0a3d4caa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6492ce80a6b24ea994d9b1e24bc43eed)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:01.043542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7af22b-f0f2-4bbc-88eb-860b0a3d4caa admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:01.068522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:01.116789 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1a7af22b-f0f2-4bbc-88eb-860b0a3d4caa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6492ce80a6b24ea994d9b1e24bc43eed'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:01.130026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1a7af22b-f0f2-4bbc-88eb-860b0a3d4caa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6492ce80a6b24ea994d9b1e24bc43eed'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:01.130829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1a7af22b-f0f2-4bbc-88eb-860b0a3d4caa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6492ce80a6b24ea994d9b1e24bc43eed'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:01.131984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1a7af22b-f0f2-4bbc-88eb-860b0a3d4caa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6492ce80a6b24ea994d9b1e24bc43eed'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:01.143375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982efedd-7a23-4b2a-b9f4-05e61715258f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:01.146418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1a7af22b-f0f2-4bbc-88eb-860b0a3d4caa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6492ce80a6b24ea994d9b1e24bc43eed'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:01.148354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:01.153231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf53f-bb50-4a92-b2e7-6f2aa6f9a919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:01.154601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf53f-bb50-4a92-b2e7-6f2aa6f9a919 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.154968 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf53f-bb50-4a92-b2e7-6f2aa6f9a919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.155255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf53f-bb50-4a92-b2e7-6f2aa6f9a919 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.155967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecf53f-bb50-4a92-b2e7-6f2aa6f9a919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:01.158037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecf53f-bb50-4a92-b2e7-6f2aa6f9a919 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:01.167767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1030/2057] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:00 2022] DELETE /identity/v3/users/6492ce80a6b24ea994d9b1e24bc43eed => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:01.175144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86296d-36f5-453b-8bf7-e2ebd8ac4192 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.175990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86296d-36f5-453b-8bf7-e2ebd8ac4192 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.175990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86296d-36f5-453b-8bf7-e2ebd8ac4192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.192765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1028/2058] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:01 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:01.196991 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf53f-bb50-4a92-b2e7-6f2aa6f9a919 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:01.198810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4cecf53f-bb50-4a92-b2e7-6f2aa6f9a919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:01.203718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-0e86296d-36f5-453b-8bf7-e2ebd8ac4192 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:01.206096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0e86296d-36f5-453b-8bf7-e2ebd8ac4192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:01.215801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf53f-bb50-4a92-b2e7-6f2aa6f9a919 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:01.285688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8400df9-3d05-4475-b956-33fce13f7b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:01.291182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1031/2059] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:01.292137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400df9-3d05-4475-b956-33fce13f7b25 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.292137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400df9-3d05-4475-b956-33fce13f7b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.292137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400df9-3d05-4475-b956-33fce13f7b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.292137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8400df9-3d05-4475-b956-33fce13f7b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:01.292137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8400df9-3d05-4475-b956-33fce13f7b25 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:01.298186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86296d-36f5-453b-8bf7-e2ebd8ac4192 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:01.299625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0e86296d-36f5-453b-8bf7-e2ebd8ac4192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:01.306427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1029/2060] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:02:01.310674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8400df9-3d05-4475-b956-33fce13f7b25 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:01.321340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8400df9-3d05-4475-b956-33fce13f7b25 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:01.326559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4698b9-0730-4089-9558-0369bd193fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:01.327414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4698b9-0730-4089-9558-0369bd193fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.327701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4698b9-0730-4089-9558-0369bd193fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.327979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4698b9-0730-4089-9558-0369bd193fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.328706 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4698b9-0730-4089-9558-0369bd193fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:01.330323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4698b9-0730-4089-9558-0369bd193fbf admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:01.343770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1030/2061] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:01.348321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4698b9-0730-4089-9558-0369bd193fbf admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:01.361453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4698b9-0730-4089-9558-0369bd193fbf admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:01.363946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86296d-36f5-453b-8bf7-e2ebd8ac4192 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:01.366990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1a82-512b-40c5-8a0e-d823d49d389f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:01.367894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1a82-512b-40c5-8a0e-d823d49d389f admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.368259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1a82-512b-40c5-8a0e-d823d49d389f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.368536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1a82-512b-40c5-8a0e-d823d49d389f admin admin] PATH_INFO: `/v3/projects/fd495d3128214b4ba60137ce00db4d69/users/7ecc6fbceffa459aa47ba7edbb04d66b/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.368660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20740436-f0eb-4bcb-8ae1-85fae5165229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:01.369318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20740436-f0eb-4bcb-8ae1-85fae5165229 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.369606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20740436-f0eb-4bcb-8ae1-85fae5165229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.369883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20740436-f0eb-4bcb-8ae1-85fae5165229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.380007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20740436-f0eb-4bcb-8ae1-85fae5165229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:01.381496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20740436-f0eb-4bcb-8ae1-85fae5165229 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:01.384442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e1a82-512b-40c5-8a0e-d823d49d389f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd495d3128214b4ba60137ce00db4d69, user_id=7ecc6fbceffa459aa47ba7edbb04d66b, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:01.388226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e1a82-512b-40c5-8a0e-d823d49d389f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:01.391918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1032/2062] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:02:01.409433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1031/2063] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:01 2022] PUT /identity/v3/projects/fd495d3128214b4ba60137ce00db4d69/users/7ecc6fbceffa459aa47ba7edbb04d66b/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:01.420375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c83b-9b23-4bd1-8905-5a0c4aa62977 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.420665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c83b-9b23-4bd1-8905-5a0c4aa62977 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.420972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c83b-9b23-4bd1-8905-5a0c4aa62977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.451319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a002c83b-9b23-4bd1-8905-5a0c4aa62977 None None] MFA Rules not processed for user `7ecc6fbceffa459aa47ba7edbb04d66b`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:01.452720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a002c83b-9b23-4bd1-8905-5a0c4aa62977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:01.533348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1033/2064] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:01.538045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1a82-512b-40c5-8a0e-d823d49d389f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:01.615902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1a82-512b-40c5-8a0e-d823d49d389f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:01.622191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03fae9-30df-483e-82d4-02d2c422b0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:01.622898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03fae9-30df-483e-82d4-02d2c422b0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.623187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03fae9-30df-483e-82d4-02d2c422b0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.623470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03fae9-30df-483e-82d4-02d2c422b0f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.624191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03fae9-30df-483e-82d4-02d2c422b0f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:01.626116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03fae9-30df-483e-82d4-02d2c422b0f1 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:01.637245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:01.657085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1032/2065] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:01 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:01.663566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002c83b-9b23-4bd1-8905-5a0c4aa62977 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:01.679145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002c83b-9b23-4bd1-8905-5a0c4aa62977 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:01.682322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03fae9-30df-483e-82d4-02d2c422b0f1 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:01.684702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4b03fae9-30df-483e-82d4-02d2c422b0f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:01.712559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:01.774102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caa3c9-633b-4976-b079-a592254c44ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:01.774823 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caa3c9-633b-4976-b079-a592254c44ab admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.775117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caa3c9-633b-4976-b079-a592254c44ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.775432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caa3c9-633b-4976-b079-a592254c44ab admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.776508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86caa3c9-633b-4976-b079-a592254c44ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:01.781130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86caa3c9-633b-4976-b079-a592254c44ab admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:01.797431 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03fae9-30df-483e-82d4-02d2c422b0f1 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:01.804970 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:01.806242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.806691 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.806876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] PATH_INFO: `/v3/projects/db45471d682b456396cd7b5c055e0a0c` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.810674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1034/2066] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:01 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:01.814228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db45471d682b456396cd7b5c055e0a0c)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:01.814928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caa3c9-633b-4976-b079-a592254c44ab admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:01.816390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:01.830254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caa3c9-633b-4976-b079-a592254c44ab admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:01.837811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009eac25-f998-4c28-882b-287785f633ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:01.838613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009eac25-f998-4c28-882b-287785f633ca admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.838899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009eac25-f998-4c28-882b-287785f633ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.839184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009eac25-f998-4c28-882b-287785f633ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.839928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009eac25-f998-4c28-882b-287785f633ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:01.842026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009eac25-f998-4c28-882b-287785f633ca admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:01.861239 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1035/2067] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:02:01.867506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009eac25-f998-4c28-882b-287785f633ca admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:01.882853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009eac25-f998-4c28-882b-287785f633ca admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:01.884451 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db45471d682b456396cd7b5c055e0a0c', 'request_id': 'req-80cae771-9f87-4b47-a75a-e2b440e0dfe8'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:01.898381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db45471d682b456396cd7b5c055e0a0c', 'request_id': 'req-80cae771-9f87-4b47-a75a-e2b440e0dfe8'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:01.913370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db45471d682b456396cd7b5c055e0a0c', 'request_id': 'req-80cae771-9f87-4b47-a75a-e2b440e0dfe8'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:01.934320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] The token cache is being invalidate because project db45471d682b456396cd7b5c055e0a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:01.934612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:01.936461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-80cae771-9f87-4b47-a75a-e2b440e0dfe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:01.939180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1033/2068] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:01 2022] DELETE /identity/v3/projects/db45471d682b456396cd7b5c055e0a0c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:01.946023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29b70c-bb51-4def-8783-dd562fb1a7a2 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.946400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29b70c-bb51-4def-8783-dd562fb1a7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.946824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29b70c-bb51-4def-8783-dd562fb1a7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.983522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-2e29b70c-bb51-4def-8783-dd562fb1a7a2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:01.984892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2e29b70c-bb51-4def-8783-dd562fb1a7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:01.985742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3be50-1e0e-4448-87dd-9719c1603897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:01.986751 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3be50-1e0e-4448-87dd-9719c1603897 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:01.987310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3be50-1e0e-4448-87dd-9719c1603897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:01.988192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3be50-1e0e-4448-87dd-9719c1603897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:01.989313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3be50-1e0e-4448-87dd-9719c1603897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:01.992124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3be50-1e0e-4448-87dd-9719c1603897 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:01.993617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1034/2069] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:02:02.003535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc91b2-6502-4108-835e-0826a95e92f3 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.003860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc91b2-6502-4108-835e-0826a95e92f3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.006876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc91b2-6502-4108-835e-0826a95e92f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.018365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1036/2070] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:02.019248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3be50-1e0e-4448-87dd-9719c1603897 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.021070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c3d3be50-1e0e-4448-87dd-9719c1603897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:02.024195 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7fcc91b2-6502-4108-835e-0826a95e92f3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:02.025468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7fcc91b2-6502-4108-835e-0826a95e92f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:02.033626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1035/2071] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:02.036150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3be50-1e0e-4448-87dd-9719c1603897 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.038665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc91b2-6502-4108-835e-0826a95e92f3 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.043843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065674b-5eaa-478d-b109-dffa88f41c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.044869 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065674b-5eaa-478d-b109-dffa88f41c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.045185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065674b-5eaa-478d-b109-dffa88f41c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.045454 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065674b-5eaa-478d-b109-dffa88f41c89 admin admin] PATH_INFO: `/v3/projects/87c32e0d89e64966bb6284a567e4cb73/users/0898907bb49b43149e331d9ce96e55ff/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.047989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc91b2-6502-4108-835e-0826a95e92f3 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.052801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3df23c-b141-4002-9176-3585795b6e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.053487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3df23c-b141-4002-9176-3585795b6e98 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.053768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3df23c-b141-4002-9176-3585795b6e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.054043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3df23c-b141-4002-9176-3585795b6e98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.054829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3df23c-b141-4002-9176-3585795b6e98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:02:02.055134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3df23c-b141-4002-9176-3585795b6e98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.056732 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3df23c-b141-4002-9176-3585795b6e98 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.061170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065674b-5eaa-478d-b109-dffa88f41c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c32e0d89e64966bb6284a567e4cb73, user_id=0898907bb49b43149e331d9ce96e55ff, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.063943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065674b-5eaa-478d-b109-dffa88f41c89 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.070607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1036/2072] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:02:02.075153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3df23c-b141-4002-9176-3585795b6e98 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.079020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1037/2073] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:02 2022] PUT /identity/v3/projects/87c32e0d89e64966bb6284a567e4cb73/users/0898907bb49b43149e331d9ce96e55ff/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:02.085404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3df23c-b141-4002-9176-3585795b6e98 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.091401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d3f20-7b3a-4b5f-9732-e6ec51a04b04 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.091691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d3f20-7b3a-4b5f-9732-e6ec51a04b04 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.091968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d3f20-7b3a-4b5f-9732-e6ec51a04b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.120915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ca9d3f20-7b3a-4b5f-9732-e6ec51a04b04 None None] MFA Rules not processed for user `0898907bb49b43149e331d9ce96e55ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:02.124010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ca9d3f20-7b3a-4b5f-9732-e6ec51a04b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:02.150915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:02.169394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5408d6-9449-41e5-8346-ec1e439fb78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.170078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5408d6-9449-41e5-8346-ec1e439fb78b admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.170391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5408d6-9449-41e5-8346-ec1e439fb78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.170672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5408d6-9449-41e5-8346-ec1e439fb78b admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.171577 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5408d6-9449-41e5-8346-ec1e439fb78b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.173548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5408d6-9449-41e5-8346-ec1e439fb78b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.208232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1037/2074] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:02 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:02:02.213399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5408d6-9449-41e5-8346-ec1e439fb78b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.246216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1038/2075] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:02:02.247106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d3f20-7b3a-4b5f-9732-e6ec51a04b04 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.253725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:02.273439 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d3f20-7b3a-4b5f-9732-e6ec51a04b04 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.338633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5408d6-9449-41e5-8346-ec1e439fb78b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.343698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea72551-39bf-4528-8671-755b510a057b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.344382 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72551-39bf-4528-8671-755b510a057b admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.344758 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72551-39bf-4528-8671-755b510a057b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.344965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72551-39bf-4528-8671-755b510a057b admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.345639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea72551-39bf-4528-8671-755b510a057b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.347721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea72551-39bf-4528-8671-755b510a057b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.360885 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc9c9b-56fb-4f79-951b-05c3de8e49fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.361780 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc9c9b-56fb-4f79-951b-05c3de8e49fe admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.362168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc9c9b-56fb-4f79-951b-05c3de8e49fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.362527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc9c9b-56fb-4f79-951b-05c3de8e49fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.363468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc9c9b-56fb-4f79-951b-05c3de8e49fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.365545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc9c9b-56fb-4f79-951b-05c3de8e49fe admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.372715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1038/2076] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:02 2022] POST /identity/v3/users => generated 284 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:02.377090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea72551-39bf-4528-8671-755b510a057b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.390488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea72551-39bf-4528-8671-755b510a057b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.398160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a86b7b-c7bf-4a18-8f32-2c64bee8f6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.398945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a86b7b-c7bf-4a18-8f32-2c64bee8f6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.399304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a86b7b-c7bf-4a18-8f32-2c64bee8f6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.399760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a86b7b-c7bf-4a18-8f32-2c64bee8f6b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.400504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a86b7b-c7bf-4a18-8f32-2c64bee8f6b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.406519 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a86b7b-c7bf-4a18-8f32-2c64bee8f6b2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.412376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1039/2077] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:02 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:02.416260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc9c9b-56fb-4f79-951b-05c3de8e49fe admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.421329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1039/2078] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:02.425205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a86b7b-c7bf-4a18-8f32-2c64bee8f6b2 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.436089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a86b7b-c7bf-4a18-8f32-2c64bee8f6b2 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.494044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc9c9b-56fb-4f79-951b-05c3de8e49fe admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.504469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233cc5-4e3d-4a43-979c-b402ec2eeab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.505208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233cc5-4e3d-4a43-979c-b402ec2eeab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.505535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233cc5-4e3d-4a43-979c-b402ec2eeab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.505782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233cc5-4e3d-4a43-979c-b402ec2eeab6 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.507197 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76233cc5-4e3d-4a43-979c-b402ec2eeab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.509450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76233cc5-4e3d-4a43-979c-b402ec2eeab6 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.520243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee04734-bbc3-46b9-bef6-2db0624a41a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.521059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee04734-bbc3-46b9-bef6-2db0624a41a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.521321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee04734-bbc3-46b9-bef6-2db0624a41a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.521584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee04734-bbc3-46b9-bef6-2db0624a41a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.523122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee04734-bbc3-46b9-bef6-2db0624a41a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.525972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee04734-bbc3-46b9-bef6-2db0624a41a9 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.536365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1040/2079] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:02 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:02:02.540738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233cc5-4e3d-4a43-979c-b402ec2eeab6 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.553901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1040/2080] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:02:02.554897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee04734-bbc3-46b9-bef6-2db0624a41a9 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.566633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee04734-bbc3-46b9-bef6-2db0624a41a9 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.571061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233cc5-4e3d-4a43-979c-b402ec2eeab6 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.572201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6841ec-2425-474b-9cee-b97ab43c6dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.573112 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6841ec-2425-474b-9cee-b97ab43c6dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.573394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6841ec-2425-474b-9cee-b97ab43c6dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.573673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6841ec-2425-474b-9cee-b97ab43c6dd9 admin admin] PATH_INFO: `/v3/projects/8b33c88ff8b1492bb228167557f4630f/users/97691a3b2ac3418993ce3af07a0baeef/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.580990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78c11f-b5cc-4e17-8cfb-8b3fef8a2a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.582030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78c11f-b5cc-4e17-8cfb-8b3fef8a2a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.582463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78c11f-b5cc-4e17-8cfb-8b3fef8a2a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.583285 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78c11f-b5cc-4e17-8cfb-8b3fef8a2a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.584264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78c11f-b5cc-4e17-8cfb-8b3fef8a2a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.586730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78c11f-b5cc-4e17-8cfb-8b3fef8a2a20 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.599462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6841ec-2425-474b-9cee-b97ab43c6dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b33c88ff8b1492bb228167557f4630f, user_id=97691a3b2ac3418993ce3af07a0baeef, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.600268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1041/2081] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:02:02.604375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78c11f-b5cc-4e17-8cfb-8b3fef8a2a20 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.604733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6841ec-2425-474b-9cee-b97ab43c6dd9 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.619135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78c11f-b5cc-4e17-8cfb-8b3fef8a2a20 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.624852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1041/2082] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:02 2022] PUT /identity/v3/projects/8b33c88ff8b1492bb228167557f4630f/users/97691a3b2ac3418993ce3af07a0baeef/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:02.643853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8d9c7-a6d3-4a13-a65d-8d57078dddc0 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.643853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8d9c7-a6d3-4a13-a65d-8d57078dddc0 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.643853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8d9c7-a6d3-4a13-a65d-8d57078dddc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.672574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-6bc8d9c7-a6d3-4a13-a65d-8d57078dddc0 None None] MFA Rules not processed for user `97691a3b2ac3418993ce3af07a0baeef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:02.673803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6bc8d9c7-a6d3-4a13-a65d-8d57078dddc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:02.688220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3dbdb-1ea5-429c-9d26-e873b6a6c478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.689143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3dbdb-1ea5-429c-9d26-e873b6a6c478 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.689424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3dbdb-1ea5-429c-9d26-e873b6a6c478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.689701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3dbdb-1ea5-429c-9d26-e873b6a6c478 admin admin] PATH_INFO: `/v3/projects/a49229ae45b1414887a1a282ab327c77/users/74a2ea9753be460fb9f0311861230d94/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.693490 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:02.702174 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:02.707269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3dbdb-1ea5-429c-9d26-e873b6a6c478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49229ae45b1414887a1a282ab327c77, user_id=74a2ea9753be460fb9f0311861230d94, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.711211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3dbdb-1ea5-429c-9d26-e873b6a6c478 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.724945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1042/2083] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:02 2022] PUT /identity/v3/projects/a49229ae45b1414887a1a282ab327c77/users/74a2ea9753be460fb9f0311861230d94/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:02.729088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3dbdb-1ea5-429c-9d26-e873b6a6c478 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.747910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1042/2084] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:02.760390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8d9c7-a6d3-4a13-a65d-8d57078dddc0 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.762054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6bc8d9c7-a6d3-4a13-a65d-8d57078dddc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:02.800255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3dbdb-1ea5-429c-9d26-e873b6a6c478 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.805537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f045678-cea8-4067-9bde-3e95c9b12686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.806301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045678-cea8-4067-9bde-3e95c9b12686 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.806672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045678-cea8-4067-9bde-3e95c9b12686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.807035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045678-cea8-4067-9bde-3e95c9b12686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.807761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f045678-cea8-4067-9bde-3e95c9b12686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.809370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f045678-cea8-4067-9bde-3e95c9b12686 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.821504 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1043/2085] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:02:02.823611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8d9c7-a6d3-4a13-a65d-8d57078dddc0 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.825263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f045678-cea8-4067-9bde-3e95c9b12686 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:02.836192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f045678-cea8-4067-9bde-3e95c9b12686 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:02.842906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a7abc-711d-4b36-823a-fb4a9f02bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.843810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a7abc-711d-4b36-823a-fb4a9f02bb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.844091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a7abc-711d-4b36-823a-fb4a9f02bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.844374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a7abc-711d-4b36-823a-fb4a9f02bb86 admin admin] PATH_INFO: `/v3/projects/a49229ae45b1414887a1a282ab327c77/users/74a2ea9753be460fb9f0311861230d94/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.847040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a7abc-711d-4b36-823a-fb4a9f02bb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49229ae45b1414887a1a282ab327c77, user_id=74a2ea9753be460fb9f0311861230d94, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.849969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a7abc-711d-4b36-823a-fb4a9f02bb86 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.863143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1044/2086] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:02 2022] PUT /identity/v3/projects/a49229ae45b1414887a1a282ab327c77/users/74a2ea9753be460fb9f0311861230d94/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:02.874347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89593317-2a7b-4ed2-8777-a34e2d40baa8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.874566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89593317-2a7b-4ed2-8777-a34e2d40baa8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.874843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89593317-2a7b-4ed2-8777-a34e2d40baa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.885248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a4ca6-d859-4a60-a1c4-c66844e988a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e47f1d65687e4cfcbce764076141702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:02.885992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a4ca6-d859-4a60-a1c4-c66844e988a9 service glance] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:02.886677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a4ca6-d859-4a60-a1c4-c66844e988a9 service glance] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:02.886818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a4ca6-d859-4a60-a1c4-c66844e988a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:02.903600 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-89593317-2a7b-4ed2-8777-a34e2d40baa8 None None] MFA Rules not processed for user `74a2ea9753be460fb9f0311861230d94`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:02.905144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-89593317-2a7b-4ed2-8777-a34e2d40baa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:02.967650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a4ca6-d859-4a60-a1c4-c66844e988a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:02.969704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a4ca6-d859-4a60-a1c4-c66844e988a9 service glance] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:02.984593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1043/2087] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:02:03.008388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1045/2088] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:02:03.027909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a4ca6-d859-4a60-a1c4-c66844e988a9 service glance] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:03.031874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-896a4ca6-d859-4a60-a1c4-c66844e988a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:03.073612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:03.147385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a4ca6-d859-4a60-a1c4-c66844e988a9 service glance] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:03.212840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19b894-6926-45ce-a359-79a0c66eb8be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:03.213516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19b894-6926-45ce-a359-79a0c66eb8be service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:03.213799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19b894-6926-45ce-a359-79a0c66eb8be service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:03.214104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19b894-6926-45ce-a359-79a0c66eb8be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:03.225773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19b894-6926-45ce-a359-79a0c66eb8be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:03.227392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19b894-6926-45ce-a359-79a0c66eb8be service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:03.246783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1044/2089] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:03.290671 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89593317-2a7b-4ed2-8777-a34e2d40baa8 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:03.300323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89593317-2a7b-4ed2-8777-a34e2d40baa8 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:03.304404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f05a75-23ec-4417-baff-cd92149b977b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:03.305115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f05a75-23ec-4417-baff-cd92149b977b service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:03.305321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f05a75-23ec-4417-baff-cd92149b977b service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:03.305675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f05a75-23ec-4417-baff-cd92149b977b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:03.307573 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d4f05a75-23ec-4417-baff-cd92149b977b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:03.381014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:03.391926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f05a75-23ec-4417-baff-cd92149b977b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:03.394922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f05a75-23ec-4417-baff-cd92149b977b service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:03.407285 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1046/2090] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:04.522461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19b894-6926-45ce-a359-79a0c66eb8be service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:04.523938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6c19b894-6926-45ce-a359-79a0c66eb8be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:04.592971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19b894-6926-45ce-a359-79a0c66eb8be service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:04.598183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf777ea4-405c-4a70-a865-a48b003d9ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:04.599134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf777ea4-405c-4a70-a865-a48b003d9ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:04.599419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf777ea4-405c-4a70-a865-a48b003d9ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:04.600013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf777ea4-405c-4a70-a865-a48b003d9ff9 admin admin] PATH_INFO: `/v3/users/39adb89399764ce0af29ceea422f8fe5` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:04.601551 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf777ea4-405c-4a70-a865-a48b003d9ff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39adb89399764ce0af29ceea422f8fe5)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:04.603655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf777ea4-405c-4a70-a865-a48b003d9ff9 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:04.659588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cf777ea4-405c-4a70-a865-a48b003d9ff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39adb89399764ce0af29ceea422f8fe5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:04.672897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cf777ea4-405c-4a70-a865-a48b003d9ff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39adb89399764ce0af29ceea422f8fe5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:04.682752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cf777ea4-405c-4a70-a865-a48b003d9ff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39adb89399764ce0af29ceea422f8fe5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:04.683493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cf777ea4-405c-4a70-a865-a48b003d9ff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39adb89399764ce0af29ceea422f8fe5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:04.685581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:04.700293 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cf777ea4-405c-4a70-a865-a48b003d9ff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39adb89399764ce0af29ceea422f8fe5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:04.707703 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1045/2091] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:04 2022] DELETE /identity/v3/users/39adb89399764ce0af29ceea422f8fe5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:04.913691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f05a75-23ec-4417-baff-cd92149b977b service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:04.936113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f05a75-23ec-4417-baff-cd92149b977b service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:05.030793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:05.032027 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:05.032580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:05.032974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] PATH_INFO: `/v3/projects/cbc76452e79c4411b4592c239245cf5d` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:05.034260 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbc76452e79c4411b4592c239245cf5d)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:05.037029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:05.132099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbc76452e79c4411b4592c239245cf5d', 'request_id': 'req-6dac0ec0-7411-4536-be59-b63bafc107c4'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:05.149813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbc76452e79c4411b4592c239245cf5d', 'request_id': 'req-6dac0ec0-7411-4536-be59-b63bafc107c4'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:05.158330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbc76452e79c4411b4592c239245cf5d', 'request_id': 'req-6dac0ec0-7411-4536-be59-b63bafc107c4'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:05.181346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] The token cache is being invalidate because project cbc76452e79c4411b4592c239245cf5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:05.182200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:05.182977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-6dac0ec0-7411-4536-be59-b63bafc107c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:05.185218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1047/2092] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:04 2022] DELETE /identity/v3/projects/cbc76452e79c4411b4592c239245cf5d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:05.190812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00343393-eba9-4cfc-8667-0272c527d031 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:05.191103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00343393-eba9-4cfc-8667-0272c527d031 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:05.191381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00343393-eba9-4cfc-8667-0272c527d031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:05.221351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-00343393-eba9-4cfc-8667-0272c527d031 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:05.222702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-00343393-eba9-4cfc-8667-0272c527d031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:05.297431 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1046/2093] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:02:05.306903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7d368-dac6-4ef3-bec4-7fd3ab5875ec None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:05.307132 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7d368-dac6-4ef3-bec4-7fd3ab5875ec None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:05.307412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7d368-dac6-4ef3-bec4-7fd3ab5875ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:05.332007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-2fd7d368-dac6-4ef3-bec4-7fd3ab5875ec None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:05.333400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2fd7d368-dac6-4ef3-bec4-7fd3ab5875ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:05.342599 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1048/2094] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:05.347318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00343393-eba9-4cfc-8667-0272c527d031 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:05.358945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00343393-eba9-4cfc-8667-0272c527d031 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:05.372142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18a171-aa1f-459e-8a6a-4063f1c5622a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:05.372841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18a171-aa1f-459e-8a6a-4063f1c5622a admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:05.373405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18a171-aa1f-459e-8a6a-4063f1c5622a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:05.373516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18a171-aa1f-459e-8a6a-4063f1c5622a admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:05.374276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18a171-aa1f-459e-8a6a-4063f1c5622a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:02:05.374686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18a171-aa1f-459e-8a6a-4063f1c5622a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:05.376310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18a171-aa1f-459e-8a6a-4063f1c5622a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:05.394319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1047/2095] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:02:05.396614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7d368-dac6-4ef3-bec4-7fd3ab5875ec None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:05.408138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7d368-dac6-4ef3-bec4-7fd3ab5875ec None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:05.414481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508a4d8-bc87-48ba-9b12-c468b6c48d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:05.414900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508a4d8-bc87-48ba-9b12-c468b6c48d6a admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:05.415188 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508a4d8-bc87-48ba-9b12-c468b6c48d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:05.415466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508a4d8-bc87-48ba-9b12-c468b6c48d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:05.416252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508a4d8-bc87-48ba-9b12-c468b6c48d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:05.418176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508a4d8-bc87-48ba-9b12-c468b6c48d6a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:05.448407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1049/2096] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:05 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:05.449747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18a171-aa1f-459e-8a6a-4063f1c5622a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:05.459632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18a171-aa1f-459e-8a6a-4063f1c5622a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:05.524378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59027111-521f-4c07-8126-99266a65f71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:05.525167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59027111-521f-4c07-8126-99266a65f71a admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:05.525525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59027111-521f-4c07-8126-99266a65f71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:05.525887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59027111-521f-4c07-8126-99266a65f71a admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:05.526766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59027111-521f-4c07-8126-99266a65f71a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:05.529155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59027111-521f-4c07-8126-99266a65f71a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:05.557669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1048/2097] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:05 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:05.559311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508a4d8-bc87-48ba-9b12-c468b6c48d6a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:05.570547 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508a4d8-bc87-48ba-9b12-c468b6c48d6a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:05.575343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e815c25-af84-40a6-a872-5e3a6de3d1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:05.576047 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815c25-af84-40a6-a872-5e3a6de3d1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:05.576330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815c25-af84-40a6-a872-5e3a6de3d1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:05.576606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815c25-af84-40a6-a872-5e3a6de3d1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:05.577339 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e815c25-af84-40a6-a872-5e3a6de3d1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:05.578988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e815c25-af84-40a6-a872-5e3a6de3d1f4 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:05.594023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1050/2098] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:05.595298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59027111-521f-4c07-8126-99266a65f71a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:05.606370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59027111-521f-4c07-8126-99266a65f71a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:05.611299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f89154-0263-45af-b9b9-c9795c611018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:05.611959 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f89154-0263-45af-b9b9-c9795c611018 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:05.612243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f89154-0263-45af-b9b9-c9795c611018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:05.612525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f89154-0263-45af-b9b9-c9795c611018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:05.613282 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f89154-0263-45af-b9b9-c9795c611018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:05.614929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f89154-0263-45af-b9b9-c9795c611018 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:05.628698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1049/2099] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:05.629937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e815c25-af84-40a6-a872-5e3a6de3d1f4 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:05.642832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e815c25-af84-40a6-a872-5e3a6de3d1f4 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:05.648264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cca4c6-7540-471c-a0dd-2b4765e47d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:05.649166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cca4c6-7540-471c-a0dd-2b4765e47d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:05.649468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cca4c6-7540-471c-a0dd-2b4765e47d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:05.649749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cca4c6-7540-471c-a0dd-2b4765e47d98 admin admin] PATH_INFO: `/v3/projects/817a68111f89402891773dea17391e94/users/7cae739c8fa74c058b5e9849ef30dca5/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:05.667115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cca4c6-7540-471c-a0dd-2b4765e47d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=817a68111f89402891773dea17391e94, user_id=7cae739c8fa74c058b5e9849ef30dca5, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:05.669810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cca4c6-7540-471c-a0dd-2b4765e47d98 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:05.682269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1051/2100] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:05 2022] PUT /identity/v3/projects/817a68111f89402891773dea17391e94/users/7cae739c8fa74c058b5e9849ef30dca5/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:05.695206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf0104-87b2-49e9-a6c6-4ee57474dc4f None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:05.695633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf0104-87b2-49e9-a6c6-4ee57474dc4f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:05.695915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf0104-87b2-49e9-a6c6-4ee57474dc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:05.724949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1fbf0104-87b2-49e9-a6c6-4ee57474dc4f None None] MFA Rules not processed for user `7cae739c8fa74c058b5e9849ef30dca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:05.726709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1fbf0104-87b2-49e9-a6c6-4ee57474dc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:05.784935 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:05.823548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1050/2101] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:05.835395 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cca4c6-7540-471c-a0dd-2b4765e47d98 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:05.837097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-23cca4c6-7540-471c-a0dd-2b4765e47d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:05.864679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:05.951590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cca4c6-7540-471c-a0dd-2b4765e47d98 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:05.957652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad91e8-6886-45f0-92d5-bb681ce73a8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:05.958330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad91e8-6886-45f0-92d5-bb681ce73a8a service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:05.958621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad91e8-6886-45f0-92d5-bb681ce73a8a service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:05.958901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad91e8-6886-45f0-92d5-bb681ce73a8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:05.967737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad91e8-6886-45f0-92d5-bb681ce73a8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:05.969335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad91e8-6886-45f0-92d5-bb681ce73a8a service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:05.978781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1052/2102] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:06.189774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf0104-87b2-49e9-a6c6-4ee57474dc4f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:06.253640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf0104-87b2-49e9-a6c6-4ee57474dc4f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:06.259099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:06.260173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:06.260483 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:06.260956 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] PATH_INFO: `/v3/users/7cae739c8fa74c058b5e9849ef30dca5` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:06.262851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cae739c8fa74c058b5e9849ef30dca5)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:06.265335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:06.312534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cae739c8fa74c058b5e9849ef30dca5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:06.324985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cae739c8fa74c058b5e9849ef30dca5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:06.332231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cae739c8fa74c058b5e9849ef30dca5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:06.332850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cae739c8fa74c058b5e9849ef30dca5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:06.340695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cae739c8fa74c058b5e9849ef30dca5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:06.346562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1051/2103] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:06 2022] DELETE /identity/v3/users/7cae739c8fa74c058b5e9849ef30dca5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:06.351303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f01e1d-fafc-4753-b1aa-b0286af53784 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:06.351601 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f01e1d-fafc-4753-b1aa-b0286af53784 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:06.351908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f01e1d-fafc-4753-b1aa-b0286af53784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:06.370037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-d5f01e1d-fafc-4753-b1aa-b0286af53784 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:06.371559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d5f01e1d-fafc-4753-b1aa-b0286af53784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:06.434811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1053/2104] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:06.446037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:06.448306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:06.522528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f01e1d-fafc-4753-b1aa-b0286af53784 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:06.524003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d5f01e1d-fafc-4753-b1aa-b0286af53784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:06.526126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f996352-f897-498e-8b1a-73a7637f946d admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:06.529840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f521744-2797-415b-9cbe-b1de74a2003b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:06.530521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f521744-2797-415b-9cbe-b1de74a2003b service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:06.530813 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f521744-2797-415b-9cbe-b1de74a2003b service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:06.531094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f521744-2797-415b-9cbe-b1de74a2003b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:06.535668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f01e1d-fafc-4753-b1aa-b0286af53784 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:06.540752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f521744-2797-415b-9cbe-b1de74a2003b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:06.542235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f521744-2797-415b-9cbe-b1de74a2003b service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:06.542860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:06.543749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:06.544025 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:06.544295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] PATH_INFO: `/v3/users/7ecc6fbceffa459aa47ba7edbb04d66b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:06.545549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ecc6fbceffa459aa47ba7edbb04d66b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:06.547619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:06.552664 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1052/2105] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:06.597082 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ecc6fbceffa459aa47ba7edbb04d66b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:06.604367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ecc6fbceffa459aa47ba7edbb04d66b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:06.605062 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ecc6fbceffa459aa47ba7edbb04d66b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:06.605711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ecc6fbceffa459aa47ba7edbb04d66b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:06.626357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ecc6fbceffa459aa47ba7edbb04d66b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:06.633812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:06.647223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1054/2106] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:06 2022] DELETE /identity/v3/users/7ecc6fbceffa459aa47ba7edbb04d66b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:06.652025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33d78-0333-43e8-8e66-866a6df8e0b6 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:06.652323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33d78-0333-43e8-8e66-866a6df8e0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:06.652634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33d78-0333-43e8-8e66-866a6df8e0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:06.673417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:06.684696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-f8b33d78-0333-43e8-8e66-866a6df8e0b6 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:06.687217 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f8b33d78-0333-43e8-8e66-866a6df8e0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:06.774478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1053/2107] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:06.784563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:06.785985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:06.868120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37567775-8af4-44e0-b8ce-d726ebcd8ab0 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:06.877739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3c0cb-d6b0-4fe8-af6b-ede478643614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:06.879176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3c0cb-d6b0-4fe8-af6b-ede478643614 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:06.879571 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3c0cb-d6b0-4fe8-af6b-ede478643614 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:06.879955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3c0cb-d6b0-4fe8-af6b-ede478643614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:06.892489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3c0cb-d6b0-4fe8-af6b-ede478643614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:06.894081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3c0cb-d6b0-4fe8-af6b-ede478643614 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:06.905355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1055/2108] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:06.919277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b33d78-0333-43e8-8e66-866a6df8e0b6 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:06.920953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f8b33d78-0333-43e8-8e66-866a6df8e0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:06.934810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b33d78-0333-43e8-8e66-866a6df8e0b6 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:06.941413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:06.942497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:06.942954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:06.943226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] PATH_INFO: `/v3/projects/817a68111f89402891773dea17391e94` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:06.945058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=817a68111f89402891773dea17391e94)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:06.950227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:07.025243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '817a68111f89402891773dea17391e94', 'request_id': 'req-378dd730-17e1-4f6a-a457-01bfb8f56d8e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.033710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '817a68111f89402891773dea17391e94', 'request_id': 'req-378dd730-17e1-4f6a-a457-01bfb8f56d8e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.039800 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '817a68111f89402891773dea17391e94', 'request_id': 'req-378dd730-17e1-4f6a-a457-01bfb8f56d8e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.050899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] The token cache is being invalidate because project 817a68111f89402891773dea17391e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:07.051184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.051848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-378dd730-17e1-4f6a-a457-01bfb8f56d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.053699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1054/2109] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:06 2022] DELETE /identity/v3/projects/817a68111f89402891773dea17391e94 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:07.062663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbf930-5b51-4cf9-b038-1aa84ba3eacf None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.063305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbf930-5b51-4cf9-b038-1aa84ba3eacf None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.063305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbf930-5b51-4cf9-b038-1aa84ba3eacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.083996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-2bdbf930-5b51-4cf9-b038-1aa84ba3eacf None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:07.085783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2bdbf930-5b51-4cf9-b038-1aa84ba3eacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:07.161055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1056/2110] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:07.162347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b3a97-4818-474e-bba8-463fdc65a701 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.162676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b3a97-4818-474e-bba8-463fdc65a701 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.162961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b3a97-4818-474e-bba8-463fdc65a701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.187888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-8b7b3a97-4818-474e-bba8-463fdc65a701 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:07.189101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8b7b3a97-4818-474e-bba8-463fdc65a701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:07.201795 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1055/2111] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:07.203599 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbf930-5b51-4cf9-b038-1aa84ba3eacf None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:07.213625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbf930-5b51-4cf9-b038-1aa84ba3eacf None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:07.219344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860c811-ab4f-495d-a8be-f8060ee69f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:07.220022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860c811-ab4f-495d-a8be-f8060ee69f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.220305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860c811-ab4f-495d-a8be-f8060ee69f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.220582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860c811-ab4f-495d-a8be-f8060ee69f87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.221318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860c811-ab4f-495d-a8be-f8060ee69f87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:02:07.221629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860c811-ab4f-495d-a8be-f8060ee69f87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:07.223283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860c811-ab4f-495d-a8be-f8060ee69f87 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:07.235477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1057/2112] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:02:07.240004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b3a97-4818-474e-bba8-463fdc65a701 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:07.251685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b3a97-4818-474e-bba8-463fdc65a701 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:07.257186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f24960-7080-4fa0-817c-313e1f088b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:07.257867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f24960-7080-4fa0-817c-313e1f088b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.258153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f24960-7080-4fa0-817c-313e1f088b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.258492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f24960-7080-4fa0-817c-313e1f088b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.259169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f24960-7080-4fa0-817c-313e1f088b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:07.261093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f24960-7080-4fa0-817c-313e1f088b6f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:07.295260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1056/2113] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:07 2022] POST /identity/v3/projects => generated 355 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:07.297008 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860c811-ab4f-495d-a8be-f8060ee69f87 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:07.310135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860c811-ab4f-495d-a8be-f8060ee69f87 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:07.316162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f24960-7080-4fa0-817c-313e1f088b6f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:07.317988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-95f24960-7080-4fa0-817c-313e1f088b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:07.377545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7fc5b-cde0-47fe-8f88-7d619a3ef254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:07.378357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7fc5b-cde0-47fe-8f88-7d619a3ef254 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.378651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7fc5b-cde0-47fe-8f88-7d619a3ef254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.379012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7fc5b-cde0-47fe-8f88-7d619a3ef254 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.379726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7fc5b-cde0-47fe-8f88-7d619a3ef254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:07.382868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7fc5b-cde0-47fe-8f88-7d619a3ef254 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:07.391881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f24960-7080-4fa0-817c-313e1f088b6f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:07.399363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:07.400368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.400713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.401062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] PATH_INFO: `/v3/projects/fd495d3128214b4ba60137ce00db4d69` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.404275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd495d3128214b4ba60137ce00db4d69)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:07.406484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:07.409436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:07.413398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1058/2114] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:07 2022] POST /identity/v3/users => generated 278 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:07.414543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7fc5b-cde0-47fe-8f88-7d619a3ef254 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:07.424646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7fc5b-cde0-47fe-8f88-7d619a3ef254 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:07.430110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac9810-0663-458e-9c20-97abb24b0a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:07.430832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac9810-0663-458e-9c20-97abb24b0a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.431120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac9810-0663-458e-9c20-97abb24b0a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.431429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac9810-0663-458e-9c20-97abb24b0a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.432164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac9810-0663-458e-9c20-97abb24b0a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:07.433820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac9810-0663-458e-9c20-97abb24b0a4d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:07.449813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1059/2115] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:02:07.449813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac9810-0663-458e-9c20-97abb24b0a4d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:07.454722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:07.460553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac9810-0663-458e-9c20-97abb24b0a4d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:07.464853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd495d3128214b4ba60137ce00db4d69', 'request_id': 'req-dbee751b-164f-484d-9e9a-b6f4e22a8a82'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.473257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd495d3128214b4ba60137ce00db4d69', 'request_id': 'req-dbee751b-164f-484d-9e9a-b6f4e22a8a82'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.480073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd495d3128214b4ba60137ce00db4d69', 'request_id': 'req-dbee751b-164f-484d-9e9a-b6f4e22a8a82'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.496556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] The token cache is being invalidate because project fd495d3128214b4ba60137ce00db4d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:07.496937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.499175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.501054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1057/2116] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:07 2022] DELETE /identity/v3/projects/fd495d3128214b4ba60137ce00db4d69 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:07.505246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:07.507833 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:07.536393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515f27f-2ee4-4787-a5ba-ba9c07b98067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:07.538105 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515f27f-2ee4-4787-a5ba-ba9c07b98067 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.538455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515f27f-2ee4-4787-a5ba-ba9c07b98067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.538830 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515f27f-2ee4-4787-a5ba-ba9c07b98067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.539815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515f27f-2ee4-4787-a5ba-ba9c07b98067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:07.541999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515f27f-2ee4-4787-a5ba-ba9c07b98067 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:07.554657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1060/2117] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:07.560108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515f27f-2ee4-4787-a5ba-ba9c07b98067 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:07.561907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7515f27f-2ee4-4787-a5ba-ba9c07b98067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:07.577037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515f27f-2ee4-4787-a5ba-ba9c07b98067 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:07.584036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886950e-c3cf-47a0-bcd1-bd156b7fccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:07.585378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886950e-c3cf-47a0-bcd1-bd156b7fccc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.585934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886950e-c3cf-47a0-bcd1-bd156b7fccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.586323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886950e-c3cf-47a0-bcd1-bd156b7fccc2 admin admin] PATH_INFO: `/v3/projects/b986f508287047c4aa3afa851f587185/users/cafa85ae4e424216918837f7c6381675/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.586839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee751b-164f-484d-9e9a-b6f4e22a8a82 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:07.592463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:07.593355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.593656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.593933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] PATH_INFO: `/v3/users/0898907bb49b43149e331d9ce96e55ff` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.595545 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0898907bb49b43149e331d9ce96e55ff)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:07.597814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:07.608285 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886950e-c3cf-47a0-bcd1-bd156b7fccc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b986f508287047c4aa3afa851f587185, user_id=cafa85ae4e424216918837f7c6381675, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:07.612589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886950e-c3cf-47a0-bcd1-bd156b7fccc2 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:07.634628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1061/2118] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:07 2022] PUT /identity/v3/projects/b986f508287047c4aa3afa851f587185/users/cafa85ae4e424216918837f7c6381675/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:07.647101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2037a-4544-4df7-848f-66072b7ce2c9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.647930 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2037a-4544-4df7-848f-66072b7ce2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.648346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2037a-4544-4df7-848f-66072b7ce2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.669488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0898907bb49b43149e331d9ce96e55ff'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.683013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0898907bb49b43149e331d9ce96e55ff'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.683741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-59b2037a-4544-4df7-848f-66072b7ce2c9 None None] MFA Rules not processed for user `cafa85ae4e424216918837f7c6381675`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:07.685192 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-59b2037a-4544-4df7-848f-66072b7ce2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:07.692655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0898907bb49b43149e331d9ce96e55ff'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.693608 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0898907bb49b43149e331d9ce96e55ff'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.702781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0898907bb49b43149e331d9ce96e55ff'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:07.709973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1058/2119] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:07 2022] DELETE /identity/v3/users/0898907bb49b43149e331d9ce96e55ff => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:07.773613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1062/2120] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:07 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:02:07.778654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:07.781676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:07.813606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:07.885966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c8147-e916-49db-b361-1b9403d86c6f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:07.894681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc3eb6-dc54-412f-b6e8-fc0a4b397acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:07.895865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc3eb6-dc54-412f-b6e8-fc0a4b397acf admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:07.895865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc3eb6-dc54-412f-b6e8-fc0a4b397acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:07.896191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc3eb6-dc54-412f-b6e8-fc0a4b397acf admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:07.899007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc3eb6-dc54-412f-b6e8-fc0a4b397acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:07.902017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc3eb6-dc54-412f-b6e8-fc0a4b397acf admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:07.941850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1059/2121] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:07 2022] POST /identity/v3/projects => generated 355 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:07.946638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2037a-4544-4df7-848f-66072b7ce2c9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:08.034622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2037a-4544-4df7-848f-66072b7ce2c9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:08.044361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:08.104361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05505bab-19c2-4136-96e4-0aafc79a10e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:08.105180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05505bab-19c2-4136-96e4-0aafc79a10e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:08.105366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05505bab-19c2-4136-96e4-0aafc79a10e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:08.105719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05505bab-19c2-4136-96e4-0aafc79a10e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:08.106507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05505bab-19c2-4136-96e4-0aafc79a10e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:08.108562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05505bab-19c2-4136-96e4-0aafc79a10e0 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:08.139799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1063/2122] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:07 2022] POST /identity/v3/users => generated 284 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:08.142392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc3eb6-dc54-412f-b6e8-fc0a4b397acf admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:08.148575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05505bab-19c2-4136-96e4-0aafc79a10e0 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:08.150400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-05505bab-19c2-4136-96e4-0aafc79a10e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:08.157136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc3eb6-dc54-412f-b6e8-fc0a4b397acf admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:08.172650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05505bab-19c2-4136-96e4-0aafc79a10e0 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:08.182609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6c338-4567-4236-afba-3e8e18f7e980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:08.183309 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6c338-4567-4236-afba-3e8e18f7e980 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:08.183601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6c338-4567-4236-afba-3e8e18f7e980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:08.183826 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:08.183946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6c338-4567-4236-afba-3e8e18f7e980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:08.184636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6c338-4567-4236-afba-3e8e18f7e980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:08.184759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:08.184972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:08.185247 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] PATH_INFO: `/v3/projects/87c32e0d89e64966bb6284a567e4cb73` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:08.186265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6c338-4567-4236-afba-3e8e18f7e980 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:08.186699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87c32e0d89e64966bb6284a567e4cb73)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:08.188635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:08.212649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1060/2123] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:08.217294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6c338-4567-4236-afba-3e8e18f7e980 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:08.230971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6c338-4567-4236-afba-3e8e18f7e980 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:08.235866 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff7bdc-8b2b-4366-b5a0-237612b0e233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:08.236883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff7bdc-8b2b-4366-b5a0-237612b0e233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:08.237044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff7bdc-8b2b-4366-b5a0-237612b0e233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:08.237315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff7bdc-8b2b-4366-b5a0-237612b0e233 admin admin] PATH_INFO: `/v3/projects/03a24fef52204594b5bf08ff54e85d4d/users/02966c2937934570876c3f04015cf389/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:08.256653 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87c32e0d89e64966bb6284a567e4cb73', 'request_id': 'req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:08.262967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff7bdc-8b2b-4366-b5a0-237612b0e233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a24fef52204594b5bf08ff54e85d4d, user_id=02966c2937934570876c3f04015cf389, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:08.266746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff7bdc-8b2b-4366-b5a0-237612b0e233 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:08.270406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87c32e0d89e64966bb6284a567e4cb73', 'request_id': 'req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:08.279126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87c32e0d89e64966bb6284a567e4cb73', 'request_id': 'req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:08.288676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1061/2124] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:08 2022] PUT /identity/v3/projects/03a24fef52204594b5bf08ff54e85d4d/users/02966c2937934570876c3f04015cf389/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:08.293958 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff7bdc-8b2b-4366-b5a0-237612b0e233 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:08.294140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] The token cache is being invalidate because project 87c32e0d89e64966bb6284a567e4cb73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:08.294140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:08.294587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:08.296585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1064/2125] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:08 2022] DELETE /identity/v3/projects/87c32e0d89e64966bb6284a567e4cb73 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:08.300491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:08.302020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:08.375560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff7bdc-8b2b-4366-b5a0-237612b0e233 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:08.380835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1c97f-e7d8-4cd3-b5e2-b8cfd367d6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:08.382794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1c97f-e7d8-4cd3-b5e2-b8cfd367d6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:08.383073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1c97f-e7d8-4cd3-b5e2-b8cfd367d6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:08.383349 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1c97f-e7d8-4cd3-b5e2-b8cfd367d6e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:08.384086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1c97f-e7d8-4cd3-b5e2-b8cfd367d6e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:08.385805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1c97f-e7d8-4cd3-b5e2-b8cfd367d6e6 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:08.389638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ff256-dc3c-4a33-a3d1-daee94bd28f3 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:08.406675 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1062/2126] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:02:08.409447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1c97f-e7d8-4cd3-b5e2-b8cfd367d6e6 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:08.410024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:08.410608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:08.410935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:08.411212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] PATH_INFO: `/v3/users/74a2ea9753be460fb9f0311861230d94` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:08.413635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a2ea9753be460fb9f0311861230d94)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:08.415847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:08.419745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-38c1c97f-e7d8-4cd3-b5e2-b8cfd367d6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:08.433370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1c97f-e7d8-4cd3-b5e2-b8cfd367d6e6 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:08.442922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512cbe4-0f27-4488-8259-d4aef5aa8fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:08.444765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512cbe4-0f27-4488-8259-d4aef5aa8fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:08.445574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512cbe4-0f27-4488-8259-d4aef5aa8fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:08.446008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512cbe4-0f27-4488-8259-d4aef5aa8fb9 admin admin] PATH_INFO: `/v3/projects/03a24fef52204594b5bf08ff54e85d4d/users/02966c2937934570876c3f04015cf389/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:08.448399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512cbe4-0f27-4488-8259-d4aef5aa8fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a24fef52204594b5bf08ff54e85d4d, user_id=02966c2937934570876c3f04015cf389, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:08.451462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512cbe4-0f27-4488-8259-d4aef5aa8fb9 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:08.468482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:08.486354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1063/2127] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:08 2022] PUT /identity/v3/projects/03a24fef52204594b5bf08ff54e85d4d/users/02966c2937934570876c3f04015cf389/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:08.488592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a2ea9753be460fb9f0311861230d94'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:08.491010 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45256a6f-2036-445a-9dea-c3ca840298a6 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:08.491299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45256a6f-2036-445a-9dea-c3ca840298a6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:08.491577 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45256a6f-2036-445a-9dea-c3ca840298a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:08.500954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a2ea9753be460fb9f0311861230d94'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:08.501696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a2ea9753be460fb9f0311861230d94'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:08.502644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a2ea9753be460fb9f0311861230d94'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:08.519680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a2ea9753be460fb9f0311861230d94'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:08.522967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-45256a6f-2036-445a-9dea-c3ca840298a6 None None] MFA Rules not processed for user `02966c2937934570876c3f04015cf389`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:08.524251 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-45256a6f-2036-445a-9dea-c3ca840298a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:08.534155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1065/2128] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:08 2022] DELETE /identity/v3/users/74a2ea9753be460fb9f0311861230d94 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:08.558958 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:08.600143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1064/2129] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:08 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:08.612350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:08.613837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:08.710132 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd04f9-2d2a-4813-b4c7-1c503f89d8d8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:08.717467 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1010e0-21e3-4ab5-a39c-c035eb50535a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:08.718360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1010e0-21e3-4ab5-a39c-c035eb50535a service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:08.718649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1010e0-21e3-4ab5-a39c-c035eb50535a service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:08.718929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1010e0-21e3-4ab5-a39c-c035eb50535a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:08.802960 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1010e0-21e3-4ab5-a39c-c035eb50535a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:08.805163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1010e0-21e3-4ab5-a39c-c035eb50535a service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:08.824058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1066/2130] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:08 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:08.863275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45256a6f-2036-445a-9dea-c3ca840298a6 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:08.866037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-45256a6f-2036-445a-9dea-c3ca840298a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:08.950910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1010e0-21e3-4ab5-a39c-c035eb50535a service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:08.959928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1010e0-21e3-4ab5-a39c-c035eb50535a service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:08.963637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c27410-c3d0-4376-90bf-748a096e60cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:08.964448 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c27410-c3d0-4376-90bf-748a096e60cd service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:08.964737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c27410-c3d0-4376-90bf-748a096e60cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:08.965147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c27410-c3d0-4376-90bf-748a096e60cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:08.966955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-27c27410-c3d0-4376-90bf-748a096e60cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:08.990435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45256a6f-2036-445a-9dea-c3ca840298a6 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:09.039584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:09.056401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c27410-c3d0-4376-90bf-748a096e60cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:09.058310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c27410-c3d0-4376-90bf-748a096e60cd service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:09.069978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1067/2131] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:08 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:02:09.076640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:09.077785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.078139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.078515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] PATH_INFO: `/v3/projects/a49229ae45b1414887a1a282ab327c77` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.079801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49229ae45b1414887a1a282ab327c77)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:09.084018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:09.143069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49229ae45b1414887a1a282ab327c77', 'request_id': 'req-c87901d7-1de4-4bbe-9492-c5a466e85327'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:09.151869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49229ae45b1414887a1a282ab327c77', 'request_id': 'req-c87901d7-1de4-4bbe-9492-c5a466e85327'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:09.158557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49229ae45b1414887a1a282ab327c77', 'request_id': 'req-c87901d7-1de4-4bbe-9492-c5a466e85327'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:09.174376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] The token cache is being invalidate because project a49229ae45b1414887a1a282ab327c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:09.174673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:09.175450 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-c87901d7-1de4-4bbe-9492-c5a466e85327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:09.177124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1065/2132] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:08 2022] DELETE /identity/v3/projects/a49229ae45b1414887a1a282ab327c77 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:09.187668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb001-2322-490d-b6b8-101001fe66e4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.188165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb001-2322-490d-b6b8-101001fe66e4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.188247 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb001-2322-490d-b6b8-101001fe66e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.210113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-027fb001-2322-490d-b6b8-101001fe66e4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:09.211363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-027fb001-2322-490d-b6b8-101001fe66e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:09.283449 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1068/2133] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:09.292341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682fdca-a33d-4206-bd28-a491d45d2c86 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.292639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682fdca-a33d-4206-bd28-a491d45d2c86 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.292924 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682fdca-a33d-4206-bd28-a491d45d2c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.306591 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:09.313582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-c682fdca-a33d-4206-bd28-a491d45d2c86 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:09.315827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c682fdca-a33d-4206-bd28-a491d45d2c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:09.324671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1066/2134] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:09.328892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fb001-2322-490d-b6b8-101001fe66e4 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:09.339357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fb001-2322-490d-b6b8-101001fe66e4 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:09.344640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a89d9-e12c-4f2d-b8d7-90f59b42b2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:09.345572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a89d9-e12c-4f2d-b8d7-90f59b42b2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.346022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a89d9-e12c-4f2d-b8d7-90f59b42b2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.346326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a89d9-e12c-4f2d-b8d7-90f59b42b2c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.347067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a89d9-e12c-4f2d-b8d7-90f59b42b2c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:02:09.347378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a89d9-e12c-4f2d-b8d7-90f59b42b2c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:09.348992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a89d9-e12c-4f2d-b8d7-90f59b42b2c8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:09.361727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1069/2135] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:02:09.366406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682fdca-a33d-4206-bd28-a491d45d2c86 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:09.375704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682fdca-a33d-4206-bd28-a491d45d2c86 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:09.380799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49971ac4-f50b-4010-ae95-6b87ea997055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:09.381948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49971ac4-f50b-4010-ae95-6b87ea997055 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.382234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49971ac4-f50b-4010-ae95-6b87ea997055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.382547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49971ac4-f50b-4010-ae95-6b87ea997055 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.383252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49971ac4-f50b-4010-ae95-6b87ea997055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:09.385178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49971ac4-f50b-4010-ae95-6b87ea997055 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:09.414836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1067/2136] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:09 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:09.416910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a89d9-e12c-4f2d-b8d7-90f59b42b2c8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:09.427620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a89d9-e12c-4f2d-b8d7-90f59b42b2c8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:09.493510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ec084-d446-4955-bb96-dafdd68260f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:09.494210 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ec084-d446-4955-bb96-dafdd68260f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.494521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ec084-d446-4955-bb96-dafdd68260f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.494799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ec084-d446-4955-bb96-dafdd68260f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.495801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ec084-d446-4955-bb96-dafdd68260f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:09.497742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ec084-d446-4955-bb96-dafdd68260f3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:09.518661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1070/2137] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:09 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:02:09.523476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49971ac4-f50b-4010-ae95-6b87ea997055 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:09.539684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49971ac4-f50b-4010-ae95-6b87ea997055 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:09.545080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715442-e0ee-48c5-8b99-881136e2e086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:09.545766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715442-e0ee-48c5-8b99-881136e2e086 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.546049 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715442-e0ee-48c5-8b99-881136e2e086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.546358 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715442-e0ee-48c5-8b99-881136e2e086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.547096 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f715442-e0ee-48c5-8b99-881136e2e086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:09.549012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f715442-e0ee-48c5-8b99-881136e2e086 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:09.559973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1068/2138] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:09.563860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ec084-d446-4955-bb96-dafdd68260f3 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:09.577184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ec084-d446-4955-bb96-dafdd68260f3 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:09.582202 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a18f4-06c4-4295-bb6e-0fe9d6bd5020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:09.582973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a18f4-06c4-4295-bb6e-0fe9d6bd5020 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.583261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a18f4-06c4-4295-bb6e-0fe9d6bd5020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.583538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a18f4-06c4-4295-bb6e-0fe9d6bd5020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.585101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a18f4-06c4-4295-bb6e-0fe9d6bd5020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:09.586823 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a18f4-06c4-4295-bb6e-0fe9d6bd5020 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:09.602963 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1071/2139] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:09.606965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715442-e0ee-48c5-8b99-881136e2e086 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:09.617495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715442-e0ee-48c5-8b99-881136e2e086 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:09.622566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268c0c9-f019-4147-8851-ffcabcb0d5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:09.623526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268c0c9-f019-4147-8851-ffcabcb0d5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.623814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268c0c9-f019-4147-8851-ffcabcb0d5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.624092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268c0c9-f019-4147-8851-ffcabcb0d5fe admin admin] PATH_INFO: `/v3/projects/4fa34f264fa24cc4952a10ca2395d8ff/users/af82b2cfacd3483fbd9f6a10a7bf8431/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.639337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268c0c9-f019-4147-8851-ffcabcb0d5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa34f264fa24cc4952a10ca2395d8ff, user_id=af82b2cfacd3483fbd9f6a10a7bf8431, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:09.642004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268c0c9-f019-4147-8851-ffcabcb0d5fe admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:09.654979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1069/2140] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:09 2022] PUT /identity/v3/projects/4fa34f264fa24cc4952a10ca2395d8ff/users/af82b2cfacd3483fbd9f6a10a7bf8431/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:09.667908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367e94c-6960-439d-aba3-4cf471475790 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.668145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367e94c-6960-439d-aba3-4cf471475790 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.668430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367e94c-6960-439d-aba3-4cf471475790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.695752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8367e94c-6960-439d-aba3-4cf471475790 None None] MFA Rules not processed for user `af82b2cfacd3483fbd9f6a10a7bf8431`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:09.696958 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8367e94c-6960-439d-aba3-4cf471475790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:09.745015 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:09.766403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1072/2141] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:09.771004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268c0c9-f019-4147-8851-ffcabcb0d5fe admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:09.834776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268c0c9-f019-4147-8851-ffcabcb0d5fe admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:09.839889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4fbc-32f5-40a1-8f55-1f7c90bd805a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:09.840563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4fbc-32f5-40a1-8f55-1f7c90bd805a admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.840846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4fbc-32f5-40a1-8f55-1f7c90bd805a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.841122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4fbc-32f5-40a1-8f55-1f7c90bd805a admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.841834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b4fbc-32f5-40a1-8f55-1f7c90bd805a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:09.843867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b4fbc-32f5-40a1-8f55-1f7c90bd805a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:09.875243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1070/2142] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:09 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:09.879270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367e94c-6960-439d-aba3-4cf471475790 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:09.892985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367e94c-6960-439d-aba3-4cf471475790 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:09.960532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bbf0c-87a5-4db7-825f-e74720cea418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:09.961916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bbf0c-87a5-4db7-825f-e74720cea418 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:09.962195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bbf0c-87a5-4db7-825f-e74720cea418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:09.962497 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bbf0c-87a5-4db7-825f-e74720cea418 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:09.963191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bbf0c-87a5-4db7-825f-e74720cea418 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:09.965139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bbf0c-87a5-4db7-825f-e74720cea418 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:09.985719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1073/2143] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:09 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:09.989678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4fbc-32f5-40a1-8f55-1f7c90bd805a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:10.000367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4fbc-32f5-40a1-8f55-1f7c90bd805a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:10.005019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62e151-37e4-4f16-aed8-ff6c164fc544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:10.005691 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e151-37e4-4f16-aed8-ff6c164fc544 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:10.005971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e151-37e4-4f16-aed8-ff6c164fc544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:10.006797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e151-37e4-4f16-aed8-ff6c164fc544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:10.007531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62e151-37e4-4f16-aed8-ff6c164fc544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:10.009129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62e151-37e4-4f16-aed8-ff6c164fc544 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:10.020877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1071/2144] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:10.025615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bbf0c-87a5-4db7-825f-e74720cea418 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:10.036111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bbf0c-87a5-4db7-825f-e74720cea418 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:10.040851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb7e74-4023-4161-8d6a-01991a8dc66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:10.042110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb7e74-4023-4161-8d6a-01991a8dc66e admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:10.042567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb7e74-4023-4161-8d6a-01991a8dc66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:10.042889 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb7e74-4023-4161-8d6a-01991a8dc66e admin admin] PATH_INFO: `/v3/projects/9c4043ddd8064adb8a97a54b4f5b3a3f/users/3d36902775ab4050b53bb0fa64396561/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:10.058146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb7e74-4023-4161-8d6a-01991a8dc66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4043ddd8064adb8a97a54b4f5b3a3f, user_id=3d36902775ab4050b53bb0fa64396561, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:10.060907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb7e74-4023-4161-8d6a-01991a8dc66e admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:10.082427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1074/2145] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:10 2022] PUT /identity/v3/projects/9c4043ddd8064adb8a97a54b4f5b3a3f/users/3d36902775ab4050b53bb0fa64396561/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:10.082560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62e151-37e4-4f16-aed8-ff6c164fc544 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:10.093232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62e151-37e4-4f16-aed8-ff6c164fc544 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:10.152843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414f414-79eb-4346-9ee0-6e92a3be82f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:10.153552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f414-79eb-4346-9ee0-6e92a3be82f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:10.153843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f414-79eb-4346-9ee0-6e92a3be82f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:10.154122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f414-79eb-4346-9ee0-6e92a3be82f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:10.154906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414f414-79eb-4346-9ee0-6e92a3be82f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:10.156563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414f414-79eb-4346-9ee0-6e92a3be82f1 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:10.168180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1072/2146] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:10.170814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb7e74-4023-4161-8d6a-01991a8dc66e admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:10.181642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb7e74-4023-4161-8d6a-01991a8dc66e admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:10.186362 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bfc02-8055-482e-bfac-574b330a868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:10.187393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bfc02-8055-482e-bfac-574b330a868d admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:10.187678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bfc02-8055-482e-bfac-574b330a868d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:10.188202 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bfc02-8055-482e-bfac-574b330a868d admin admin] PATH_INFO: `/v3/projects/9c4043ddd8064adb8a97a54b4f5b3a3f/users/3d36902775ab4050b53bb0fa64396561/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:10.190041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bfc02-8055-482e-bfac-574b330a868d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4043ddd8064adb8a97a54b4f5b3a3f, user_id=3d36902775ab4050b53bb0fa64396561, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:10.192928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bfc02-8055-482e-bfac-574b330a868d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:10.204787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1075/2147] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:10 2022] PUT /identity/v3/projects/9c4043ddd8064adb8a97a54b4f5b3a3f/users/3d36902775ab4050b53bb0fa64396561/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:10.215036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0dbdb-4e81-4d7d-9a36-a48d2f75e480 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:10.215326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0dbdb-4e81-4d7d-9a36-a48d2f75e480 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:10.215610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0dbdb-4e81-4d7d-9a36-a48d2f75e480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:10.242847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:10.246648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-d7f0dbdb-4e81-4d7d-9a36-a48d2f75e480 None None] MFA Rules not processed for user `3d36902775ab4050b53bb0fa64396561`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:10.249427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d7f0dbdb-4e81-4d7d-9a36-a48d2f75e480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:10.331339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1073/2148] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:10.345861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bfc02-8055-482e-bfac-574b330a868d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:10.347906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-077bfc02-8055-482e-bfac-574b330a868d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:10.418447 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bfc02-8055-482e-bfac-574b330a868d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:10.422217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9647315-2795-4a68-8cf9-e4739b321878 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:10.422900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9647315-2795-4a68-8cf9-e4739b321878 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:10.423188 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9647315-2795-4a68-8cf9-e4739b321878 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:10.423467 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9647315-2795-4a68-8cf9-e4739b321878 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:10.433021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9647315-2795-4a68-8cf9-e4739b321878 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:10.434451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9647315-2795-4a68-8cf9-e4739b321878 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:10.444429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1076/2149] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:10.517183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0dbdb-4e81-4d7d-9a36-a48d2f75e480 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:10.526708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0dbdb-4e81-4d7d-9a36-a48d2f75e480 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:10.530261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a4fc1-0c02-44a3-aadd-e577d5aeefe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:10.530926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a4fc1-0c02-44a3-aadd-e577d5aeefe7 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:10.531211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a4fc1-0c02-44a3-aadd-e577d5aeefe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:10.531488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a4fc1-0c02-44a3-aadd-e577d5aeefe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:10.587763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a4fc1-0c02-44a3-aadd-e577d5aeefe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:10.589311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a4fc1-0c02-44a3-aadd-e577d5aeefe7 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:10.598562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1074/2150] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:10.864348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9647315-2795-4a68-8cf9-e4739b321878 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:10.909375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:10.928510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9647315-2795-4a68-8cf9-e4739b321878 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:10.933099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:10.935526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:10.935526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:10.935526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] PATH_INFO: `/v3/users/af82b2cfacd3483fbd9f6a10a7bf8431` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:10.936687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af82b2cfacd3483fbd9f6a10a7bf8431)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:10.938857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:10.994376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af82b2cfacd3483fbd9f6a10a7bf8431'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:11.004621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af82b2cfacd3483fbd9f6a10a7bf8431'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:11.005281 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af82b2cfacd3483fbd9f6a10a7bf8431'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:11.005973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af82b2cfacd3483fbd9f6a10a7bf8431'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:11.021187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af82b2cfacd3483fbd9f6a10a7bf8431'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:11.035430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1077/2151] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:10 2022] DELETE /identity/v3/users/af82b2cfacd3483fbd9f6a10a7bf8431 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:11.039988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75eccf-eaa8-4de2-b60c-ddf539be94e2 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:11.040447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75eccf-eaa8-4de2-b60c-ddf539be94e2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:11.040683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75eccf-eaa8-4de2-b60c-ddf539be94e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:11.058560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5c75eccf-eaa8-4de2-b60c-ddf539be94e2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:11.059758 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5c75eccf-eaa8-4de2-b60c-ddf539be94e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:11.130682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1075/2152] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:11.141793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:11.143309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:11.208501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdb7af-1160-4ff1-86df-02e2d33d2325 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:11.216758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c08ebb-23dc-4095-9b12-5e9991939d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:11.217729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c08ebb-23dc-4095-9b12-5e9991939d2d service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:11.218082 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c08ebb-23dc-4095-9b12-5e9991939d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:11.218498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c08ebb-23dc-4095-9b12-5e9991939d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:11.228751 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c08ebb-23dc-4095-9b12-5e9991939d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:11.230211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c08ebb-23dc-4095-9b12-5e9991939d2d service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:11.240415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1078/2153] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:11.608108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75eccf-eaa8-4de2-b60c-ddf539be94e2 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:11.609748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5c75eccf-eaa8-4de2-b60c-ddf539be94e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:11.635107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75eccf-eaa8-4de2-b60c-ddf539be94e2 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:11.643749 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:11.644956 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:11.645317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:11.645605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] PATH_INFO: `/v3/projects/4fa34f264fa24cc4952a10ca2395d8ff` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:11.651263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa34f264fa24cc4952a10ca2395d8ff)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:11.653321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:11.706928 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:11.731366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa34f264fa24cc4952a10ca2395d8ff', 'request_id': 'req-bb2b31ae-3f07-4e78-a463-a178d8781222'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:11.742306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa34f264fa24cc4952a10ca2395d8ff', 'request_id': 'req-bb2b31ae-3f07-4e78-a463-a178d8781222'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:11.758207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa34f264fa24cc4952a10ca2395d8ff', 'request_id': 'req-bb2b31ae-3f07-4e78-a463-a178d8781222'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:11.774878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] The token cache is being invalidate because project 4fa34f264fa24cc4952a10ca2395d8ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:11.775150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:11.775844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:11.777683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1076/2154] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:11 2022] DELETE /identity/v3/projects/4fa34f264fa24cc4952a10ca2395d8ff => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:11.781472 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c08ebb-23dc-4095-9b12-5e9991939d2d service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:11.801578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c08ebb-23dc-4095-9b12-5e9991939d2d service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:11.932300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473519be-f3fb-40e5-9f45-e9e075e2432b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:11.933640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473519be-f3fb-40e5-9f45-e9e075e2432b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:11.934155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473519be-f3fb-40e5-9f45-e9e075e2432b admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:11.934621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473519be-f3fb-40e5-9f45-e9e075e2432b admin admin] PATH_INFO: `/v3/users/3d36902775ab4050b53bb0fa64396561` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:11.936141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473519be-f3fb-40e5-9f45-e9e075e2432b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d36902775ab4050b53bb0fa64396561)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:11.938348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473519be-f3fb-40e5-9f45-e9e075e2432b admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:11.944705 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:12.024906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-473519be-f3fb-40e5-9f45-e9e075e2432b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d36902775ab4050b53bb0fa64396561'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:12.037467 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-473519be-f3fb-40e5-9f45-e9e075e2432b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d36902775ab4050b53bb0fa64396561'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:12.038122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-473519be-f3fb-40e5-9f45-e9e075e2432b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d36902775ab4050b53bb0fa64396561'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:12.039051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-473519be-f3fb-40e5-9f45-e9e075e2432b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d36902775ab4050b53bb0fa64396561'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:12.055360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-473519be-f3fb-40e5-9f45-e9e075e2432b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d36902775ab4050b53bb0fa64396561'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:12.067808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1079/2155] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:11 2022] DELETE /identity/v3/users/3d36902775ab4050b53bb0fa64396561 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:12.350252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:12.352371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:12.371750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b31ae-3f07-4e78-a463-a178d8781222 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:12.440614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:12.441975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:12.442887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:12.443502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] PATH_INFO: `/v3/projects/9c4043ddd8064adb8a97a54b4f5b3a3f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:12.445029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4043ddd8064adb8a97a54b4f5b3a3f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:12.447092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:12.496742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4043ddd8064adb8a97a54b4f5b3a3f', 'request_id': 'req-9ba19fcf-027e-4a2d-b459-8f28770dd413'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:12.504995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4043ddd8064adb8a97a54b4f5b3a3f', 'request_id': 'req-9ba19fcf-027e-4a2d-b459-8f28770dd413'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:12.512789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4043ddd8064adb8a97a54b4f5b3a3f', 'request_id': 'req-9ba19fcf-027e-4a2d-b459-8f28770dd413'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:12.524650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] The token cache is being invalidate because project 9c4043ddd8064adb8a97a54b4f5b3a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:12.524866 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:12.525586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-9ba19fcf-027e-4a2d-b459-8f28770dd413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:12.527890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1077/2156] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:12 2022] DELETE /identity/v3/projects/9c4043ddd8064adb8a97a54b4f5b3a3f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:12.532722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ce198-2d8f-4436-954d-94f004f87350 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:12.533210 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ce198-2d8f-4436-954d-94f004f87350 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:12.533302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ce198-2d8f-4436-954d-94f004f87350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:12.552252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-855ce198-2d8f-4436-954d-94f004f87350 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:12.554037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-855ce198-2d8f-4436-954d-94f004f87350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:12.620957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1080/2157] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:12.621307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31e65f-11c2-477b-84aa-ce987e32484f None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:12.621531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31e65f-11c2-477b-84aa-ce987e32484f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:12.622018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31e65f-11c2-477b-84aa-ce987e32484f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:12.628407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5e125-3c75-468f-94ac-41d3734fa046 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:12.628728 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5e125-3c75-468f-94ac-41d3734fa046 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:12.629012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5e125-3c75-468f-94ac-41d3734fa046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:12.644642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-4e31e65f-11c2-477b-84aa-ce987e32484f None None] MFA Rules not processed for user `226c8c0918714060ac908316c7effc08`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:12.645942 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4e31e65f-11c2-477b-84aa-ce987e32484f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:12.649575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e5e5e125-3c75-468f-94ac-41d3734fa046 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:12.651385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e5e5e125-3c75-468f-94ac-41d3734fa046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:12.660118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1081/2158] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:02:12.664224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5e125-3c75-468f-94ac-41d3734fa046 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:12.673262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5e125-3c75-468f-94ac-41d3734fa046 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:12.678709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b91a7-1f92-4a29-9dae-2532f91ccd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:12.679523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b91a7-1f92-4a29-9dae-2532f91ccd1c admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:12.679811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b91a7-1f92-4a29-9dae-2532f91ccd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:12.680090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b91a7-1f92-4a29-9dae-2532f91ccd1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:12.680816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b91a7-1f92-4a29-9dae-2532f91ccd1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:02:12.681753 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b91a7-1f92-4a29-9dae-2532f91ccd1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:12.683494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b91a7-1f92-4a29-9dae-2532f91ccd1c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:12.697564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1082/2159] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:02:12.698849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:12.702100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b91a7-1f92-4a29-9dae-2532f91ccd1c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:12.712180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1078/2160] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:12.714791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b91a7-1f92-4a29-9dae-2532f91ccd1c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:12.720811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76435da7-e5a8-4fb0-839a-e7e474f05208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:12.721625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76435da7-e5a8-4fb0-839a-e7e474f05208 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:12.722398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76435da7-e5a8-4fb0-839a-e7e474f05208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:12.722398 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76435da7-e5a8-4fb0-839a-e7e474f05208 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:12.722937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76435da7-e5a8-4fb0-839a-e7e474f05208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:12.724917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76435da7-e5a8-4fb0-839a-e7e474f05208 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:12.766567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1083/2161] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:12 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:12.770673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31e65f-11c2-477b-84aa-ce987e32484f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:12.783501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31e65f-11c2-477b-84aa-ce987e32484f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:12.850884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d911df3-f1f9-4fe7-b2a6-95b025374365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:12.851682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911df3-f1f9-4fe7-b2a6-95b025374365 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:12.852169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911df3-f1f9-4fe7-b2a6-95b025374365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:12.852541 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911df3-f1f9-4fe7-b2a6-95b025374365 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:12.853259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d911df3-f1f9-4fe7-b2a6-95b025374365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:12.855282 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d911df3-f1f9-4fe7-b2a6-95b025374365 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:12.875825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1079/2162] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:12 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:12.881682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76435da7-e5a8-4fb0-839a-e7e474f05208 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:12.892607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76435da7-e5a8-4fb0-839a-e7e474f05208 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:12.897977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5367e4b-eaac-4baf-98f2-f0eac80d5418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:12.898741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5367e4b-eaac-4baf-98f2-f0eac80d5418 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:12.899030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5367e4b-eaac-4baf-98f2-f0eac80d5418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:12.899382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5367e4b-eaac-4baf-98f2-f0eac80d5418 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:12.900162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5367e4b-eaac-4baf-98f2-f0eac80d5418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:12.901861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5367e4b-eaac-4baf-98f2-f0eac80d5418 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:12.914712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1084/2163] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:12.918546 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d911df3-f1f9-4fe7-b2a6-95b025374365 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:12.928762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d911df3-f1f9-4fe7-b2a6-95b025374365 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:12.933627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9e166-98c3-47a6-ae0e-b0ef7d699afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:12.934319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9e166-98c3-47a6-ae0e-b0ef7d699afb admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:12.934662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9e166-98c3-47a6-ae0e-b0ef7d699afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:12.935037 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9e166-98c3-47a6-ae0e-b0ef7d699afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:12.935977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9e166-98c3-47a6-ae0e-b0ef7d699afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:12.937660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9e166-98c3-47a6-ae0e-b0ef7d699afb admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:12.951061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1080/2164] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:02:12.955151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5367e4b-eaac-4baf-98f2-f0eac80d5418 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:12.966146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5367e4b-eaac-4baf-98f2-f0eac80d5418 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:12.970876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d678e6-fb13-471b-b8f5-956edf8a30dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:12.971765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d678e6-fb13-471b-b8f5-956edf8a30dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:12.972050 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d678e6-fb13-471b-b8f5-956edf8a30dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:12.972321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d678e6-fb13-471b-b8f5-956edf8a30dd admin admin] PATH_INFO: `/v3/projects/63a9569ea11443af8ac16813d1d9b557/users/e24a409763cd4b759d8b7f534b532afb/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:12.990106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d678e6-fb13-471b-b8f5-956edf8a30dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a9569ea11443af8ac16813d1d9b557, user_id=e24a409763cd4b759d8b7f534b532afb, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:12.992940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d678e6-fb13-471b-b8f5-956edf8a30dd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:13.009865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1085/2165] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:12 2022] PUT /identity/v3/projects/63a9569ea11443af8ac16813d1d9b557/users/e24a409763cd4b759d8b7f534b532afb/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:13.020886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d88a-ecdb-41f0-bdc2-2a2413cc3a5a None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:13.021159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d88a-ecdb-41f0-bdc2-2a2413cc3a5a None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:13.021438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d88a-ecdb-41f0-bdc2-2a2413cc3a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:13.058375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-b490d88a-ecdb-41f0-bdc2-2a2413cc3a5a None None] MFA Rules not processed for user `e24a409763cd4b759d8b7f534b532afb`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:13.059612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b490d88a-ecdb-41f0-bdc2-2a2413cc3a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:13.182661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1081/2166] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:13.196599 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d678e6-fb13-471b-b8f5-956edf8a30dd admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:13.199725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-46d678e6-fb13-471b-b8f5-956edf8a30dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:13.215103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:13.296305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d678e6-fb13-471b-b8f5-956edf8a30dd admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:13.303993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3e1e5-447d-450e-8224-6dbc12bcb22f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:13.304774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3e1e5-447d-450e-8224-6dbc12bcb22f service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:13.305101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3e1e5-447d-450e-8224-6dbc12bcb22f service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:13.305942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3e1e5-447d-450e-8224-6dbc12bcb22f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:13.318587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3e1e5-447d-450e-8224-6dbc12bcb22f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:13.320038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3e1e5-447d-450e-8224-6dbc12bcb22f service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:13.337866 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1086/2167] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:13.378024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490d88a-ecdb-41f0-bdc2-2a2413cc3a5a None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:13.495705 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490d88a-ecdb-41f0-bdc2-2a2413cc3a5a None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:13.501271 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:13.502713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:13.503097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:13.503474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] PATH_INFO: `/v3/users/e24a409763cd4b759d8b7f534b532afb` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:13.505084 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e24a409763cd4b759d8b7f534b532afb)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:13.507280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:13.512692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:13.565159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e24a409763cd4b759d8b7f534b532afb'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:13.583604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e24a409763cd4b759d8b7f534b532afb'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:13.593318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e24a409763cd4b759d8b7f534b532afb'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:13.594262 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e24a409763cd4b759d8b7f534b532afb'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:13.608254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e24a409763cd4b759d8b7f534b532afb'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:13.616372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1082/2168] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:13 2022] DELETE /identity/v3/users/e24a409763cd4b759d8b7f534b532afb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:13.621131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d2aa6-1e96-4fff-9dad-dfcaaea61cd0 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:13.621598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d2aa6-1e96-4fff-9dad-dfcaaea61cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:13.621686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d2aa6-1e96-4fff-9dad-dfcaaea61cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:13.642731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-af2d2aa6-1e96-4fff-9dad-dfcaaea61cd0 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:13.644059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-af2d2aa6-1e96-4fff-9dad-dfcaaea61cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:13.716359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1087/2169] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:13.726549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:13.728286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:13.802888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad678f3-80b2-4506-8293-3713b0be4ba5 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:13.807076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0f007-3045-4de3-8cfb-9ecfa3a129cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:13.807752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0f007-3045-4de3-8cfb-9ecfa3a129cd service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:13.808185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0f007-3045-4de3-8cfb-9ecfa3a129cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:13.808308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0f007-3045-4de3-8cfb-9ecfa3a129cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:13.819928 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0f007-3045-4de3-8cfb-9ecfa3a129cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:13.821358 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0f007-3045-4de3-8cfb-9ecfa3a129cd service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:13.832476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1083/2170] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:14.148955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d2aa6-1e96-4fff-9dad-dfcaaea61cd0 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:14.150820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-af2d2aa6-1e96-4fff-9dad-dfcaaea61cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:14.164411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d2aa6-1e96-4fff-9dad-dfcaaea61cd0 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:14.169947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:14.172485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:14.172832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:14.173113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] PATH_INFO: `/v3/projects/63a9569ea11443af8ac16813d1d9b557` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:14.174374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a9569ea11443af8ac16813d1d9b557)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:14.176474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:14.245063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a9569ea11443af8ac16813d1d9b557', 'request_id': 'req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:14.254556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a9569ea11443af8ac16813d1d9b557', 'request_id': 'req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:14.261085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a9569ea11443af8ac16813d1d9b557', 'request_id': 'req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:14.276732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] The token cache is being invalidate because project 63a9569ea11443af8ac16813d1d9b557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:14.277024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:14.277691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6c013d-4cbc-4d68-84e3-5a8bd5c20a16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:14.279336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1088/2171] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:14 2022] DELETE /identity/v3/projects/63a9569ea11443af8ac16813d1d9b557 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:14.283894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49003b-2128-4338-bb07-4185e79061ad None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:14.284187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49003b-2128-4338-bb07-4185e79061ad None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:14.284470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49003b-2128-4338-bb07-4185e79061ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:14.306331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-df49003b-2128-4338-bb07-4185e79061ad None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:14.307684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-df49003b-2128-4338-bb07-4185e79061ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:14.391129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1084/2172] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:02:14.401461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110a91b-0afe-481b-b996-8bef43a32331 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:14.401896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110a91b-0afe-481b-b996-8bef43a32331 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:14.402377 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110a91b-0afe-481b-b996-8bef43a32331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:14.423266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a110a91b-0afe-481b-b996-8bef43a32331 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:14.424554 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a110a91b-0afe-481b-b996-8bef43a32331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:14.434341 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1089/2173] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:14.438447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49003b-2128-4338-bb07-4185e79061ad None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:14.449753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49003b-2128-4338-bb07-4185e79061ad None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:14.455011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4485e-83b0-4915-8d2f-0406a344197b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:14.455871 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4485e-83b0-4915-8d2f-0406a344197b admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:14.456273 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4485e-83b0-4915-8d2f-0406a344197b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:14.456735 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4485e-83b0-4915-8d2f-0406a344197b admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:14.457534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4485e-83b0-4915-8d2f-0406a344197b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:02:14.459195 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4485e-83b0-4915-8d2f-0406a344197b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:14.459712 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4485e-83b0-4915-8d2f-0406a344197b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:14.490125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1085/2174] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:02:14.492849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110a91b-0afe-481b-b996-8bef43a32331 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:14.508204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110a91b-0afe-481b-b996-8bef43a32331 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:14.513961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c363b7-6ce2-49f0-a693-5785864d212c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:14.514803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c363b7-6ce2-49f0-a693-5785864d212c admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:14.515075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c363b7-6ce2-49f0-a693-5785864d212c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:14.515351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c363b7-6ce2-49f0-a693-5785864d212c admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:14.516044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c363b7-6ce2-49f0-a693-5785864d212c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:14.518116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c363b7-6ce2-49f0-a693-5785864d212c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:14.553245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1090/2175] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:14 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:14.554454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4485e-83b0-4915-8d2f-0406a344197b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:14.586738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4485e-83b0-4915-8d2f-0406a344197b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:14.606964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:14.673032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da7c67-fb2d-44bb-a9c5-8520d65535e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:14.674015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da7c67-fb2d-44bb-a9c5-8520d65535e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:14.674437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da7c67-fb2d-44bb-a9c5-8520d65535e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:14.674818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da7c67-fb2d-44bb-a9c5-8520d65535e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:14.675766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da7c67-fb2d-44bb-a9c5-8520d65535e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:14.677853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da7c67-fb2d-44bb-a9c5-8520d65535e5 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:14.710103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1086/2176] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:14 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:14.714632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c363b7-6ce2-49f0-a693-5785864d212c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:14.721456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:14.726769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c363b7-6ce2-49f0-a693-5785864d212c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:14.738118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed9109-699c-4ec5-b855-c1940610b142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:14.740178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9109-699c-4ec5-b855-c1940610b142 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:14.741455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9109-699c-4ec5-b855-c1940610b142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:14.742110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9109-699c-4ec5-b855-c1940610b142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:14.743879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed9109-699c-4ec5-b855-c1940610b142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:14.746724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed9109-699c-4ec5-b855-c1940610b142 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:14.758984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1091/2177] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:02:14.765004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da7c67-fb2d-44bb-a9c5-8520d65535e5 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:14.777069 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da7c67-fb2d-44bb-a9c5-8520d65535e5 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:14.782080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1c175-d08e-4f3c-bf7d-4fe85c566b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:14.782788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1c175-d08e-4f3c-bf7d-4fe85c566b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:14.783067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1c175-d08e-4f3c-bf7d-4fe85c566b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:14.783351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1c175-d08e-4f3c-bf7d-4fe85c566b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:14.784086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1c175-d08e-4f3c-bf7d-4fe85c566b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:14.785744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1c175-d08e-4f3c-bf7d-4fe85c566b14 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:14.797313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1087/2178] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:14.801330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed9109-699c-4ec5-b855-c1940610b142 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:14.816024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed9109-699c-4ec5-b855-c1940610b142 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:14.821236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7003688-12f9-4148-b8c1-51e10bb59915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:14.821679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7003688-12f9-4148-b8c1-51e10bb59915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:14.821962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7003688-12f9-4148-b8c1-51e10bb59915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:14.822238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7003688-12f9-4148-b8c1-51e10bb59915 admin admin] PATH_INFO: `/v3/projects/a622cffda8044b5c837866bb725c8f4e/users/5f7ebc64271e4e2aa1aa8a2a8baf8ef9/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:14.840273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7003688-12f9-4148-b8c1-51e10bb59915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a622cffda8044b5c837866bb725c8f4e, user_id=5f7ebc64271e4e2aa1aa8a2a8baf8ef9, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:14.843072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7003688-12f9-4148-b8c1-51e10bb59915 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:14.860095 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1092/2179] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:14 2022] PUT /identity/v3/projects/a622cffda8044b5c837866bb725c8f4e/users/5f7ebc64271e4e2aa1aa8a2a8baf8ef9/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:14.871558 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850474c-64af-42d2-ba9e-5250e62688cb None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:14.871858 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850474c-64af-42d2-ba9e-5250e62688cb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:14.872143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850474c-64af-42d2-ba9e-5250e62688cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:14.907504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5850474c-64af-42d2-ba9e-5250e62688cb None None] MFA Rules not processed for user `5f7ebc64271e4e2aa1aa8a2a8baf8ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:14.908777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5850474c-64af-42d2-ba9e-5250e62688cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:15.000904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1088/2180] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:15.006107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7003688-12f9-4148-b8c1-51e10bb59915 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:15.075905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7003688-12f9-4148-b8c1-51e10bb59915 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:15.081902 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9aec89-fe28-4c3c-8b33-f4598ae3deb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:15.082611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9aec89-fe28-4c3c-8b33-f4598ae3deb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:15.083014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9aec89-fe28-4c3c-8b33-f4598ae3deb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:15.083271 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9aec89-fe28-4c3c-8b33-f4598ae3deb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:15.084128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9aec89-fe28-4c3c-8b33-f4598ae3deb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:15.086116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9aec89-fe28-4c3c-8b33-f4598ae3deb0 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:15.116194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1093/2181] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:15 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:02:15.121055 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850474c-64af-42d2-ba9e-5250e62688cb None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:15.132685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850474c-64af-42d2-ba9e-5250e62688cb None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:15.197240 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcd8f3-30c7-4c90-bbc9-274d99bf79d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:15.198004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcd8f3-30c7-4c90-bbc9-274d99bf79d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:15.198309 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcd8f3-30c7-4c90-bbc9-274d99bf79d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:15.198597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcd8f3-30c7-4c90-bbc9-274d99bf79d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:15.199540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcd8f3-30c7-4c90-bbc9-274d99bf79d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:15.201477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcd8f3-30c7-4c90-bbc9-274d99bf79d5 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:15.226235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1089/2182] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:15 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:15.228158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9aec89-fe28-4c3c-8b33-f4598ae3deb0 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:15.241981 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9aec89-fe28-4c3c-8b33-f4598ae3deb0 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:15.247918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178c478-c740-4bb7-af8f-9126854dca22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:15.249747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178c478-c740-4bb7-af8f-9126854dca22 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:15.250223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178c478-c740-4bb7-af8f-9126854dca22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:15.250538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178c478-c740-4bb7-af8f-9126854dca22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:15.251292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178c478-c740-4bb7-af8f-9126854dca22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:15.252940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178c478-c740-4bb7-af8f-9126854dca22 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:15.273299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1094/2183] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:15.278051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcd8f3-30c7-4c90-bbc9-274d99bf79d5 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:15.291155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcd8f3-30c7-4c90-bbc9-274d99bf79d5 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:15.302049 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324e313-a9e9-448d-834b-acad8d649ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:15.304320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324e313-a9e9-448d-834b-acad8d649ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:15.304634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324e313-a9e9-448d-834b-acad8d649ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:15.304944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324e313-a9e9-448d-834b-acad8d649ed4 admin admin] PATH_INFO: `/v3/projects/19f5b86af60641c088238ace04b35835/users/1ea4cdf25d0a4a7f8baaa0ae5b2c74d7/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:15.315201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:15.323190 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324e313-a9e9-448d-834b-acad8d649ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f5b86af60641c088238ace04b35835, user_id=1ea4cdf25d0a4a7f8baaa0ae5b2c74d7, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:15.326514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324e313-a9e9-448d-834b-acad8d649ed4 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:15.340992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1090/2184] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:15 2022] PUT /identity/v3/projects/19f5b86af60641c088238ace04b35835/users/1ea4cdf25d0a4a7f8baaa0ae5b2c74d7/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:15.344950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178c478-c740-4bb7-af8f-9126854dca22 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:15.356368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178c478-c740-4bb7-af8f-9126854dca22 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:15.422246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41690b-e7ca-4bc2-8351-51b39ef34a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:15.422946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41690b-e7ca-4bc2-8351-51b39ef34a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:15.423227 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41690b-e7ca-4bc2-8351-51b39ef34a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:15.423502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41690b-e7ca-4bc2-8351-51b39ef34a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:15.424229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41690b-e7ca-4bc2-8351-51b39ef34a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:15.425950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41690b-e7ca-4bc2-8351-51b39ef34a19 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:15.438984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1095/2185] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:15.440956 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324e313-a9e9-448d-834b-acad8d649ed4 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:15.454271 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324e313-a9e9-448d-834b-acad8d649ed4 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:15.460380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf889f0-85a9-484d-b8b0-f217b07f65f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:15.461495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf889f0-85a9-484d-b8b0-f217b07f65f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:15.461495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf889f0-85a9-484d-b8b0-f217b07f65f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:15.461711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf889f0-85a9-484d-b8b0-f217b07f65f2 admin admin] PATH_INFO: `/v3/projects/19f5b86af60641c088238ace04b35835/users/1ea4cdf25d0a4a7f8baaa0ae5b2c74d7/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:15.463798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf889f0-85a9-484d-b8b0-f217b07f65f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f5b86af60641c088238ace04b35835, user_id=1ea4cdf25d0a4a7f8baaa0ae5b2c74d7, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:15.466499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf889f0-85a9-484d-b8b0-f217b07f65f2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:15.477473 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1091/2186] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:15 2022] PUT /identity/v3/projects/19f5b86af60641c088238ace04b35835/users/1ea4cdf25d0a4a7f8baaa0ae5b2c74d7/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:15.487278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b835d8-f205-4c7a-902a-f6119ec2454b None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:15.487917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b835d8-f205-4c7a-902a-f6119ec2454b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:15.487917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b835d8-f205-4c7a-902a-f6119ec2454b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:15.530854 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f1b835d8-f205-4c7a-902a-f6119ec2454b None None] MFA Rules not processed for user `1ea4cdf25d0a4a7f8baaa0ae5b2c74d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:15.533658 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f1b835d8-f205-4c7a-902a-f6119ec2454b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:15.563670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:15.612955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1096/2187] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:15.629377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf889f0-85a9-484d-b8b0-f217b07f65f2 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:15.631495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9bf889f0-85a9-484d-b8b0-f217b07f65f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:15.700561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf889f0-85a9-484d-b8b0-f217b07f65f2 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:15.708786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6682b0-b4f1-4de3-ac5f-79fe7b3598cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:15.710579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6682b0-b4f1-4de3-ac5f-79fe7b3598cb service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:15.710870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6682b0-b4f1-4de3-ac5f-79fe7b3598cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:15.711151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6682b0-b4f1-4de3-ac5f-79fe7b3598cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:15.790393 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6682b0-b4f1-4de3-ac5f-79fe7b3598cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:15.792062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6682b0-b4f1-4de3-ac5f-79fe7b3598cb service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:15.801450 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1092/2188] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:19.447501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b835d8-f205-4c7a-902a-f6119ec2454b None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:19.515469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b835d8-f205-4c7a-902a-f6119ec2454b None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:19.520019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:19.520891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:19.521172 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:19.521958 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] PATH_INFO: `/v3/users/5f7ebc64271e4e2aa1aa8a2a8baf8ef9` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:19.523185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f7ebc64271e4e2aa1aa8a2a8baf8ef9)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:19.525124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:19.575751 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f7ebc64271e4e2aa1aa8a2a8baf8ef9'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:19.584432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f7ebc64271e4e2aa1aa8a2a8baf8ef9'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:19.585178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f7ebc64271e4e2aa1aa8a2a8baf8ef9'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:19.585879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f7ebc64271e4e2aa1aa8a2a8baf8ef9'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:19.598207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f7ebc64271e4e2aa1aa8a2a8baf8ef9'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:19.614555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1097/2189] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:19 2022] DELETE /identity/v3/users/5f7ebc64271e4e2aa1aa8a2a8baf8ef9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:19.619473 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b251c18-ad66-4775-8c81-3dd9494d7644 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:19.619946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b251c18-ad66-4775-8c81-3dd9494d7644 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:19.620305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b251c18-ad66-4775-8c81-3dd9494d7644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:19.637812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9b251c18-ad66-4775-8c81-3dd9494d7644 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:19.639066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9b251c18-ad66-4775-8c81-3dd9494d7644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:19.699000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1093/2190] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:19.708591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:19.710012 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:19.757589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:19.769553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28d907-e89d-4ddf-b55d-ea7e5b40eee8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:19.774892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5b70-8d4d-4f00-91f4-c83e1967dffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:19.775548 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5b70-8d4d-4f00-91f4-c83e1967dffd service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:19.775832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5b70-8d4d-4f00-91f4-c83e1967dffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:19.776102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5b70-8d4d-4f00-91f4-c83e1967dffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:19.785565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f5b70-8d4d-4f00-91f4-c83e1967dffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:19.787080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f5b70-8d4d-4f00-91f4-c83e1967dffd service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:19.796886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1098/2191] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:20.077655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b251c18-ad66-4775-8c81-3dd9494d7644 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:20.080789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9b251c18-ad66-4775-8c81-3dd9494d7644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:20.094338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b251c18-ad66-4775-8c81-3dd9494d7644 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:20.099351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:20.100304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:20.100647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:20.100997 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] PATH_INFO: `/v3/projects/a622cffda8044b5c837866bb725c8f4e` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:20.102539 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a622cffda8044b5c837866bb725c8f4e)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:20.104827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:20.135571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:20.164030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a622cffda8044b5c837866bb725c8f4e', 'request_id': 'req-ed1f0b86-c619-4296-8fa8-136c79a20d8f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.171471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a622cffda8044b5c837866bb725c8f4e', 'request_id': 'req-ed1f0b86-c619-4296-8fa8-136c79a20d8f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.177383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a622cffda8044b5c837866bb725c8f4e', 'request_id': 'req-ed1f0b86-c619-4296-8fa8-136c79a20d8f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.194486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] The token cache is being invalidate because project a622cffda8044b5c837866bb725c8f4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:20.194814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.195737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.198524 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1094/2192] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:20 2022] DELETE /identity/v3/projects/a622cffda8044b5c837866bb725c8f4e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:20.202630 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5b70-8d4d-4f00-91f4-c83e1967dffd service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:20.213052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5b70-8d4d-4f00-91f4-c83e1967dffd service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:20.297989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f3624-cc78-495b-9be0-4313f9233c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:20.299100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3624-cc78-495b-9be0-4313f9233c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:20.299490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3624-cc78-495b-9be0-4313f9233c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:20.299905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3624-cc78-495b-9be0-4313f9233c0b admin admin] PATH_INFO: `/v3/users/1ea4cdf25d0a4a7f8baaa0ae5b2c74d7` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:20.302308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f3624-cc78-495b-9be0-4313f9233c0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea4cdf25d0a4a7f8baaa0ae5b2c74d7)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:20.306183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f3624-cc78-495b-9be0-4313f9233c0b admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:20.391030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-d32f3624-cc78-495b-9be0-4313f9233c0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea4cdf25d0a4a7f8baaa0ae5b2c74d7'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.407388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-d32f3624-cc78-495b-9be0-4313f9233c0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea4cdf25d0a4a7f8baaa0ae5b2c74d7'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.409795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-d32f3624-cc78-495b-9be0-4313f9233c0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea4cdf25d0a4a7f8baaa0ae5b2c74d7'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.410738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-d32f3624-cc78-495b-9be0-4313f9233c0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea4cdf25d0a4a7f8baaa0ae5b2c74d7'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.425986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-d32f3624-cc78-495b-9be0-4313f9233c0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea4cdf25d0a4a7f8baaa0ae5b2c74d7'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.445401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1099/2193] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:20 2022] DELETE /identity/v3/users/1ea4cdf25d0a4a7f8baaa0ae5b2c74d7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:20.780687 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:20.782190 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:20.795237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f0b86-c619-4296-8fa8-136c79a20d8f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:20.852772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:20.853650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:20.853938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:20.854213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] PATH_INFO: `/v3/projects/19f5b86af60641c088238ace04b35835` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:20.855464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f5b86af60641c088238ace04b35835)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:20.857391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:20.910656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f5b86af60641c088238ace04b35835', 'request_id': 'req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.920743 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f5b86af60641c088238ace04b35835', 'request_id': 'req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.927729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f5b86af60641c088238ace04b35835', 'request_id': 'req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.939827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] The token cache is being invalidate because project 19f5b86af60641c088238ace04b35835 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:20.940114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.940787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-91e2beaa-b774-4fcc-ad94-82fce59e0dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:20.942532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1095/2194] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:20 2022] DELETE /identity/v3/projects/19f5b86af60641c088238ace04b35835 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:20.948471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55633a6-a3b7-4e5e-93b8-507e5322c6da None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:20.948944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55633a6-a3b7-4e5e-93b8-507e5322c6da None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:20.949033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55633a6-a3b7-4e5e-93b8-507e5322c6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:20.976183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a55633a6-a3b7-4e5e-93b8-507e5322c6da None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:20.977323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a55633a6-a3b7-4e5e-93b8-507e5322c6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:21.049097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1100/2195] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:21.056807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa452a8-1624-40af-b9de-482cb05ad3c1 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:21.057099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa452a8-1624-40af-b9de-482cb05ad3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:21.057381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa452a8-1624-40af-b9de-482cb05ad3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:21.077755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5aa452a8-1624-40af-b9de-482cb05ad3c1 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:21.079427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5aa452a8-1624-40af-b9de-482cb05ad3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:21.088502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1096/2196] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:21.093126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55633a6-a3b7-4e5e-93b8-507e5322c6da None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:21.104996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55633a6-a3b7-4e5e-93b8-507e5322c6da None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:21.109682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd090fa-47dc-4a14-98b6-2bc8db03b879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:21.110384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd090fa-47dc-4a14-98b6-2bc8db03b879 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:21.110677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd090fa-47dc-4a14-98b6-2bc8db03b879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:21.111022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd090fa-47dc-4a14-98b6-2bc8db03b879 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:21.112254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd090fa-47dc-4a14-98b6-2bc8db03b879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:02:21.112582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd090fa-47dc-4a14-98b6-2bc8db03b879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:21.114734 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd090fa-47dc-4a14-98b6-2bc8db03b879 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:21.117552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:21.134683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1101/2197] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:02:21.137357 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa452a8-1624-40af-b9de-482cb05ad3c1 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:21.148277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa452a8-1624-40af-b9de-482cb05ad3c1 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:21.153713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f12365-29df-4ff7-9425-9350e10a5bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:21.154494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f12365-29df-4ff7-9425-9350e10a5bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:21.154850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f12365-29df-4ff7-9425-9350e10a5bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:21.155200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f12365-29df-4ff7-9425-9350e10a5bc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:21.155989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f12365-29df-4ff7-9425-9350e10a5bc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:21.158005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f12365-29df-4ff7-9425-9350e10a5bc7 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:21.192750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1097/2198] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:21 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:02:21.197044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd090fa-47dc-4a14-98b6-2bc8db03b879 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:21.207596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd090fa-47dc-4a14-98b6-2bc8db03b879 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:21.270806 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0954fe-bdc6-4197-9ea8-bc903f3a65a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:21.271641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0954fe-bdc6-4197-9ea8-bc903f3a65a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:21.271974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0954fe-bdc6-4197-9ea8-bc903f3a65a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:21.272227 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0954fe-bdc6-4197-9ea8-bc903f3a65a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:21.272927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0954fe-bdc6-4197-9ea8-bc903f3a65a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:21.274897 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0954fe-bdc6-4197-9ea8-bc903f3a65a6 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:21.300674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1102/2199] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:21 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:02:21.302007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f12365-29df-4ff7-9425-9350e10a5bc7 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:21.314902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f12365-29df-4ff7-9425-9350e10a5bc7 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:21.321098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e7821-a341-4cd6-b62c-453f1acc4c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:21.321929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e7821-a341-4cd6-b62c-453f1acc4c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:21.322500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e7821-a341-4cd6-b62c-453f1acc4c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:21.322995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e7821-a341-4cd6-b62c-453f1acc4c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:21.325097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e7821-a341-4cd6-b62c-453f1acc4c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:21.327451 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e7821-a341-4cd6-b62c-453f1acc4c91 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:21.339476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1098/2200] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:21.343507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0954fe-bdc6-4197-9ea8-bc903f3a65a6 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:21.353679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0954fe-bdc6-4197-9ea8-bc903f3a65a6 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:21.358419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c3c01-af67-40ba-8537-32d7bad151f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:21.359085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c3c01-af67-40ba-8537-32d7bad151f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:21.359372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c3c01-af67-40ba-8537-32d7bad151f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:21.359666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c3c01-af67-40ba-8537-32d7bad151f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:21.360392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c3c01-af67-40ba-8537-32d7bad151f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:21.361990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c3c01-af67-40ba-8537-32d7bad151f7 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:21.374401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1103/2201] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:21.378631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e7821-a341-4cd6-b62c-453f1acc4c91 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:21.390378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:21.397845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e7821-a341-4cd6-b62c-453f1acc4c91 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:21.411619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e42b34-febc-4a74-860f-37d7b1335af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:21.412476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e42b34-febc-4a74-860f-37d7b1335af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:21.412804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e42b34-febc-4a74-860f-37d7b1335af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:21.413130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e42b34-febc-4a74-860f-37d7b1335af3 admin admin] PATH_INFO: `/v3/projects/97d4002620de4768b87ebf323f7cbb43/users/29c019e09e664381b5f62f01308f1010/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:21.429740 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e42b34-febc-4a74-860f-37d7b1335af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d4002620de4768b87ebf323f7cbb43, user_id=29c019e09e664381b5f62f01308f1010, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:21.432534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e42b34-febc-4a74-860f-37d7b1335af3 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:21.447203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1099/2202] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:21 2022] PUT /identity/v3/projects/97d4002620de4768b87ebf323f7cbb43/users/29c019e09e664381b5f62f01308f1010/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:21.461166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63676e-0e41-4247-b692-d0ee89d8a28a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:21.461538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63676e-0e41-4247-b692-d0ee89d8a28a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:21.462356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63676e-0e41-4247-b692-d0ee89d8a28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:21.492185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7b63676e-0e41-4247-b692-d0ee89d8a28a None None] MFA Rules not processed for user `29c019e09e664381b5f62f01308f1010`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:21.493359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7b63676e-0e41-4247-b692-d0ee89d8a28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:21.557351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1104/2203] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:21.570820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e42b34-febc-4a74-860f-37d7b1335af3 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:21.573105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e5e42b34-febc-4a74-860f-37d7b1335af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:21.699883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e42b34-febc-4a74-860f-37d7b1335af3 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:21.714611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b361a-9a18-42aa-9bdf-c977866278c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:21.715297 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b361a-9a18-42aa-9bdf-c977866278c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:21.715582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b361a-9a18-42aa-9bdf-c977866278c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:21.715859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b361a-9a18-42aa-9bdf-c977866278c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:21.737857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b361a-9a18-42aa-9bdf-c977866278c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:21.737857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b361a-9a18-42aa-9bdf-c977866278c6 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:21.746329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1100/2204] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:31.558353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:31.741185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=35,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:34.166985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63676e-0e41-4247-b692-d0ee89d8a28a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:34.168387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7b63676e-0e41-4247-b692-d0ee89d8a28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:34.243884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63676e-0e41-4247-b692-d0ee89d8a28a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:34.243884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:34.243884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:34.243884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:34.244245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] PATH_INFO: `/v3/users/cafa85ae4e424216918837f7c6381675` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:34.244245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cafa85ae4e424216918837f7c6381675)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:34.245829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:34.295353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cafa85ae4e424216918837f7c6381675'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:34.302770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cafa85ae4e424216918837f7c6381675'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:34.303396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cafa85ae4e424216918837f7c6381675'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:34.303996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cafa85ae4e424216918837f7c6381675'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:34.316081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cafa85ae4e424216918837f7c6381675'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:34.328673 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1105/2205] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:34 2022] DELETE /identity/v3/users/cafa85ae4e424216918837f7c6381675 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:34.333128 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab6f8-d242-49b2-ab61-59c1873be4ac None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:34.334164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab6f8-d242-49b2-ab61-59c1873be4ac None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:34.334164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab6f8-d242-49b2-ab61-59c1873be4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:34.351852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-2dbab6f8-d242-49b2-ab61-59c1873be4ac None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:34.353094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2dbab6f8-d242-49b2-ab61-59c1873be4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:34.429597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1101/2206] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:34.439326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:34.440805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:34.503690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b1b5b-0849-4ceb-9b77-7186ff99cf1c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:34.507971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067def9-9ad5-4bad-a5cb-f28cd27710d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:34.508625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067def9-9ad5-4bad-a5cb-f28cd27710d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:34.508905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067def9-9ad5-4bad-a5cb-f28cd27710d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:34.509177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067def9-9ad5-4bad-a5cb-f28cd27710d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:34.518541 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067def9-9ad5-4bad-a5cb-f28cd27710d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:34.519990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067def9-9ad5-4bad-a5cb-f28cd27710d1 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:34.530353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1106/2207] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:34.828275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbab6f8-d242-49b2-ab61-59c1873be4ac None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:34.830182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2dbab6f8-d242-49b2-ab61-59c1873be4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:34.852772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbab6f8-d242-49b2-ab61-59c1873be4ac None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:34.859585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:34.861077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:34.861386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:34.861763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] PATH_INFO: `/v3/projects/b986f508287047c4aa3afa851f587185` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:34.863424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b986f508287047c4aa3afa851f587185)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:34.865734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:34.924914 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b986f508287047c4aa3afa851f587185', 'request_id': 'req-f7952873-aefc-4853-9326-2158ef9cda3a'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:34.931578 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b986f508287047c4aa3afa851f587185', 'request_id': 'req-f7952873-aefc-4853-9326-2158ef9cda3a'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:34.936996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b986f508287047c4aa3afa851f587185', 'request_id': 'req-f7952873-aefc-4853-9326-2158ef9cda3a'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:34.948188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] The token cache is being invalidate because project b986f508287047c4aa3afa851f587185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:34.948478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:34.949107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:34.950721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1102/2208] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:34 2022] DELETE /identity/v3/projects/b986f508287047c4aa3afa851f587185 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:34.954636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067def9-9ad5-4bad-a5cb-f28cd27710d1 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:34.964846 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067def9-9ad5-4bad-a5cb-f28cd27710d1 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:34.983007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:35.022906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c524ca3-816a-4ff4-a73d-2f8ee64fd216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:35.023778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c524ca3-816a-4ff4-a73d-2f8ee64fd216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:35.024063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c524ca3-816a-4ff4-a73d-2f8ee64fd216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:35.024338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c524ca3-816a-4ff4-a73d-2f8ee64fd216 admin admin] PATH_INFO: `/v3/users/02966c2937934570876c3f04015cf389` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:35.025524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c524ca3-816a-4ff4-a73d-2f8ee64fd216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02966c2937934570876c3f04015cf389)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:35.027692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c524ca3-816a-4ff4-a73d-2f8ee64fd216 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:35.072685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4c524ca3-816a-4ff4-a73d-2f8ee64fd216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02966c2937934570876c3f04015cf389'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:35.079655 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4c524ca3-816a-4ff4-a73d-2f8ee64fd216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02966c2937934570876c3f04015cf389'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:35.080306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4c524ca3-816a-4ff4-a73d-2f8ee64fd216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02966c2937934570876c3f04015cf389'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:35.080909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4c524ca3-816a-4ff4-a73d-2f8ee64fd216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02966c2937934570876c3f04015cf389'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:35.093276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4c524ca3-816a-4ff4-a73d-2f8ee64fd216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02966c2937934570876c3f04015cf389'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:35.105632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1107/2209] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:34 2022] DELETE /identity/v3/users/02966c2937934570876c3f04015cf389 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:35.373574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:35.375097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:35.399166 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7952873-aefc-4853-9326-2158ef9cda3a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:35.459606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:35.460461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:35.460743 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:35.461013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] PATH_INFO: `/v3/projects/03a24fef52204594b5bf08ff54e85d4d` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:35.462192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=03a24fef52204594b5bf08ff54e85d4d)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:35.464342 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:35.494497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:35.512379 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03a24fef52204594b5bf08ff54e85d4d', 'request_id': 'req-292b38eb-fff4-4f66-9083-12c1af965e3d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:35.519099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03a24fef52204594b5bf08ff54e85d4d', 'request_id': 'req-292b38eb-fff4-4f66-9083-12c1af965e3d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:35.526818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03a24fef52204594b5bf08ff54e85d4d', 'request_id': 'req-292b38eb-fff4-4f66-9083-12c1af965e3d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:35.539756 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] The token cache is being invalidate because project 03a24fef52204594b5bf08ff54e85d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:35.540044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:35.540693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-292b38eb-fff4-4f66-9083-12c1af965e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:35.542261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1103/2210] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:35 2022] DELETE /identity/v3/projects/03a24fef52204594b5bf08ff54e85d4d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:35.547120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cc89d-b816-4486-b77c-5d8d3018feb6 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:35.547688 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cc89d-b816-4486-b77c-5d8d3018feb6 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:35.550474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cc89d-b816-4486-b77c-5d8d3018feb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:35.569176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-4a5cc89d-b816-4486-b77c-5d8d3018feb6 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:35.570375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4a5cc89d-b816-4486-b77c-5d8d3018feb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:35.629809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1108/2211] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:35.638207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686b1b-5cac-40c8-b338-65d4d3b739e3 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:35.638539 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686b1b-5cac-40c8-b338-65d4d3b739e3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:35.638894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686b1b-5cac-40c8-b338-65d4d3b739e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:35.657823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-fc686b1b-5cac-40c8-b338-65d4d3b739e3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:35.659066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-fc686b1b-5cac-40c8-b338-65d4d3b739e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:35.667208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1104/2212] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:35.671416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cc89d-b816-4486-b77c-5d8d3018feb6 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:35.681041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cc89d-b816-4486-b77c-5d8d3018feb6 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:35.685666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec62666-f335-4d89-a3cb-ff6bbd859216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:35.686322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec62666-f335-4d89-a3cb-ff6bbd859216 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:35.686609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec62666-f335-4d89-a3cb-ff6bbd859216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:35.687551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec62666-f335-4d89-a3cb-ff6bbd859216 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:35.688325 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec62666-f335-4d89-a3cb-ff6bbd859216 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:02:35.688645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec62666-f335-4d89-a3cb-ff6bbd859216 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:35.690246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec62666-f335-4d89-a3cb-ff6bbd859216 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:35.704562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1109/2213] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:02:35.706902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc686b1b-5cac-40c8-b338-65d4d3b739e3 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:35.716315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc686b1b-5cac-40c8-b338-65d4d3b739e3 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:35.720960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c67684-d427-4c5d-a088-98908ee243bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:35.721611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c67684-d427-4c5d-a088-98908ee243bc admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:35.721899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c67684-d427-4c5d-a088-98908ee243bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:35.722175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c67684-d427-4c5d-a088-98908ee243bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:35.722907 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c67684-d427-4c5d-a088-98908ee243bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:35.725346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c67684-d427-4c5d-a088-98908ee243bc admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:35.750576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1105/2214] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:35 2022] POST /identity/v3/projects => generated 387 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:02:35.754386 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec62666-f335-4d89-a3cb-ff6bbd859216 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:35.764312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec62666-f335-4d89-a3cb-ff6bbd859216 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:35.790259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:35.820836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19971727-cf39-4b6e-8e9c-86d2a41c8951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:35.821495 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19971727-cf39-4b6e-8e9c-86d2a41c8951 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:35.821778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19971727-cf39-4b6e-8e9c-86d2a41c8951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:35.822052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19971727-cf39-4b6e-8e9c-86d2a41c8951 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:35.822772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19971727-cf39-4b6e-8e9c-86d2a41c8951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:35.824689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19971727-cf39-4b6e-8e9c-86d2a41c8951 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:35.845404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1110/2215] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:35 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:35.850116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c67684-d427-4c5d-a088-98908ee243bc admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:35.861310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c67684-d427-4c5d-a088-98908ee243bc admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:35.866187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c7023-85cc-432d-81e6-714223a8405b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:35.867022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c7023-85cc-432d-81e6-714223a8405b admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:35.867311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c7023-85cc-432d-81e6-714223a8405b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:35.867589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c7023-85cc-432d-81e6-714223a8405b admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:35.868372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c7023-85cc-432d-81e6-714223a8405b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:35.870236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c7023-85cc-432d-81e6-714223a8405b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:35.884659 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1106/2216] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:35.886170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19971727-cf39-4b6e-8e9c-86d2a41c8951 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:35.896631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19971727-cf39-4b6e-8e9c-86d2a41c8951 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:35.901170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e0ecf-cfc1-4f73-a445-cabc9e8e30d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:35.901818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e0ecf-cfc1-4f73-a445-cabc9e8e30d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:35.902098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e0ecf-cfc1-4f73-a445-cabc9e8e30d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:35.902397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e0ecf-cfc1-4f73-a445-cabc9e8e30d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:35.903113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e0ecf-cfc1-4f73-a445-cabc9e8e30d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:35.904699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e0ecf-cfc1-4f73-a445-cabc9e8e30d8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:35.915266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1111/2217] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:35.919298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c7023-85cc-432d-81e6-714223a8405b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:35.929464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c7023-85cc-432d-81e6-714223a8405b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:35.934126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e208df0-b32f-42e5-9d68-121872579689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:35.935036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e208df0-b32f-42e5-9d68-121872579689 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:35.935322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e208df0-b32f-42e5-9d68-121872579689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:35.935600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e208df0-b32f-42e5-9d68-121872579689 admin admin] PATH_INFO: `/v3/projects/e31b116f7afb4907a9bb45e1736a3bf4/users/90bf25d003914bad962e64a8ac141755/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:35.951574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e208df0-b32f-42e5-9d68-121872579689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e31b116f7afb4907a9bb45e1736a3bf4, user_id=90bf25d003914bad962e64a8ac141755, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:35.954223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e208df0-b32f-42e5-9d68-121872579689 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:35.968338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1107/2218] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:35 2022] PUT /identity/v3/projects/e31b116f7afb4907a9bb45e1736a3bf4/users/90bf25d003914bad962e64a8ac141755/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:35.980377 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f38c02-a019-48f9-b8cb-84dd08b40b78 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:35.980674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f38c02-a019-48f9-b8cb-84dd08b40b78 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:35.981010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f38c02-a019-48f9-b8cb-84dd08b40b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:36.007759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b6f38c02-a019-48f9-b8cb-84dd08b40b78 None None] MFA Rules not processed for user `90bf25d003914bad962e64a8ac141755`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:36.008971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b6f38c02-a019-48f9-b8cb-84dd08b40b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:36.074788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1112/2219] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:36.081191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e208df0-b32f-42e5-9d68-121872579689 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:36.142570 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e208df0-b32f-42e5-9d68-121872579689 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:36.147726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954e8bb-fab0-4f2e-a170-93c479d11028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:36.148376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e8bb-fab0-4f2e-a170-93c479d11028 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:36.148663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e8bb-fab0-4f2e-a170-93c479d11028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:36.148940 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e8bb-fab0-4f2e-a170-93c479d11028 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:36.149659 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954e8bb-fab0-4f2e-a170-93c479d11028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:36.151586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954e8bb-fab0-4f2e-a170-93c479d11028 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:36.175537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1108/2220] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:36 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:36.179781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f38c02-a019-48f9-b8cb-84dd08b40b78 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:36.190412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f38c02-a019-48f9-b8cb-84dd08b40b78 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:36.246075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ee5ad-1e8d-4d86-895c-9b7683a907a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:36.246781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ee5ad-1e8d-4d86-895c-9b7683a907a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:36.247068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ee5ad-1e8d-4d86-895c-9b7683a907a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:36.247346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ee5ad-1e8d-4d86-895c-9b7683a907a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:36.248045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ee5ad-1e8d-4d86-895c-9b7683a907a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:36.250029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ee5ad-1e8d-4d86-895c-9b7683a907a6 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:36.271273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1113/2221] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:36 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:36.275702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954e8bb-fab0-4f2e-a170-93c479d11028 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:36.285912 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954e8bb-fab0-4f2e-a170-93c479d11028 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:36.291031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead9680-e426-43bd-ad35-de46d8ae629a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:36.291688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9680-e426-43bd-ad35-de46d8ae629a admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:36.291974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9680-e426-43bd-ad35-de46d8ae629a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:36.292253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9680-e426-43bd-ad35-de46d8ae629a admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:36.292981 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead9680-e426-43bd-ad35-de46d8ae629a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:36.294619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead9680-e426-43bd-ad35-de46d8ae629a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:36.306047 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1109/2222] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:36.309967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ee5ad-1e8d-4d86-895c-9b7683a907a6 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:36.320644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ee5ad-1e8d-4d86-895c-9b7683a907a6 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:36.325324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba40092-e16c-4b60-a4ea-3308838e5399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:36.326235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba40092-e16c-4b60-a4ea-3308838e5399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:36.326564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba40092-e16c-4b60-a4ea-3308838e5399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:36.326845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba40092-e16c-4b60-a4ea-3308838e5399 admin admin] PATH_INFO: `/v3/projects/61d14dd3157044fb838328e701423f05/users/f802d63d71b14e57b2789917754abb87/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:36.341726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba40092-e16c-4b60-a4ea-3308838e5399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d14dd3157044fb838328e701423f05, user_id=f802d63d71b14e57b2789917754abb87, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:36.344480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba40092-e16c-4b60-a4ea-3308838e5399 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:36.357767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1114/2223] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:36 2022] PUT /identity/v3/projects/61d14dd3157044fb838328e701423f05/users/f802d63d71b14e57b2789917754abb87/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:36.361757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead9680-e426-43bd-ad35-de46d8ae629a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:36.372634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead9680-e426-43bd-ad35-de46d8ae629a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:36.392781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:36.431320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda30cc-2971-4337-9069-09c2283a557e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:36.431990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda30cc-2971-4337-9069-09c2283a557e admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:36.432274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda30cc-2971-4337-9069-09c2283a557e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:36.432551 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda30cc-2971-4337-9069-09c2283a557e admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:36.433278 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda30cc-2971-4337-9069-09c2283a557e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:36.434927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda30cc-2971-4337-9069-09c2283a557e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:36.444996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1110/2224] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:36.449254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba40092-e16c-4b60-a4ea-3308838e5399 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:36.454658 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:36.461440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba40092-e16c-4b60-a4ea-3308838e5399 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:36.466310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c14ea0-3565-40ba-818f-6504818564f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:36.467248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c14ea0-3565-40ba-818f-6504818564f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:36.467532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c14ea0-3565-40ba-818f-6504818564f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:36.467804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c14ea0-3565-40ba-818f-6504818564f8 admin admin] PATH_INFO: `/v3/projects/61d14dd3157044fb838328e701423f05/users/f802d63d71b14e57b2789917754abb87/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:36.469809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c14ea0-3565-40ba-818f-6504818564f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d14dd3157044fb838328e701423f05, user_id=f802d63d71b14e57b2789917754abb87, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:36.472535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c14ea0-3565-40ba-818f-6504818564f8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:36.482681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1115/2225] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:36 2022] PUT /identity/v3/projects/61d14dd3157044fb838328e701423f05/users/f802d63d71b14e57b2789917754abb87/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:36.493100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b3ade-d261-4b5f-b5f4-2bfd995aebd8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:36.493391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b3ade-d261-4b5f-b5f4-2bfd995aebd8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:36.493668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b3ade-d261-4b5f-b5f4-2bfd995aebd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:36.520350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7c8b3ade-d261-4b5f-b5f4-2bfd995aebd8 None None] MFA Rules not processed for user `f802d63d71b14e57b2789917754abb87`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:36.521541 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7c8b3ade-d261-4b5f-b5f4-2bfd995aebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:36.586686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1111/2226] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:36.597811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c14ea0-3565-40ba-818f-6504818564f8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:36.599589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c9c14ea0-3565-40ba-818f-6504818564f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:36.657859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c14ea0-3565-40ba-818f-6504818564f8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:36.665464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756c295-aafc-4b98-b25d-7fd474e51a47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:36.666131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756c295-aafc-4b98-b25d-7fd474e51a47 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:36.666451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756c295-aafc-4b98-b25d-7fd474e51a47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:36.666741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756c295-aafc-4b98-b25d-7fd474e51a47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:36.680326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756c295-aafc-4b98-b25d-7fd474e51a47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:36.681778 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756c295-aafc-4b98-b25d-7fd474e51a47 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:36.698400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1116/2227] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:36.764264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b3ade-d261-4b5f-b5f4-2bfd995aebd8 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:36.829769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b3ade-d261-4b5f-b5f4-2bfd995aebd8 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:36.834715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:36.835592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:36.835875 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:36.836150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] PATH_INFO: `/v3/users/90bf25d003914bad962e64a8ac141755` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:36.837345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90bf25d003914bad962e64a8ac141755)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:36.839430 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:36.889438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90bf25d003914bad962e64a8ac141755'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:36.890843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:36.906711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90bf25d003914bad962e64a8ac141755'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:36.915393 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90bf25d003914bad962e64a8ac141755'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:36.916086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90bf25d003914bad962e64a8ac141755'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:36.932102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90bf25d003914bad962e64a8ac141755'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:36.933536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1112/2228] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:36 2022] DELETE /identity/v3/users/90bf25d003914bad962e64a8ac141755 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:36.937940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3241aaa-33ef-4bfd-8c8b-853d435b1a4a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:36.938232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3241aaa-33ef-4bfd-8c8b-853d435b1a4a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:36.938553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3241aaa-33ef-4bfd-8c8b-853d435b1a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:36.955518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c3241aaa-33ef-4bfd-8c8b-853d435b1a4a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:36.956701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c3241aaa-33ef-4bfd-8c8b-853d435b1a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:37.017847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1117/2229] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:37.029739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:37.031177 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:37.090066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65ed01-50a8-4fde-8afc-32fe4280c1d3 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:37.093745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640b418-a3fd-450d-8365-24fb91956401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:37.094680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640b418-a3fd-450d-8365-24fb91956401 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:37.094974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640b418-a3fd-450d-8365-24fb91956401 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:37.095258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640b418-a3fd-450d-8365-24fb91956401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:37.104552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640b418-a3fd-450d-8365-24fb91956401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:37.105991 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640b418-a3fd-450d-8365-24fb91956401 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:37.115513 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1113/2230] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:02:37.480306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3241aaa-33ef-4bfd-8c8b-853d435b1a4a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:37.481877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c3241aaa-33ef-4bfd-8c8b-853d435b1a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:37.495177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3241aaa-33ef-4bfd-8c8b-853d435b1a4a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:37.499799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:37.500669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:37.500956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:37.501229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] PATH_INFO: `/v3/projects/e31b116f7afb4907a9bb45e1736a3bf4` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:37.502450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e31b116f7afb4907a9bb45e1736a3bf4)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:37.504390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:37.550516 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e31b116f7afb4907a9bb45e1736a3bf4', 'request_id': 'req-0a288277-9bc8-47d3-9d27-5b609d208b2b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:37.557989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e31b116f7afb4907a9bb45e1736a3bf4', 'request_id': 'req-0a288277-9bc8-47d3-9d27-5b609d208b2b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:37.563519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e31b116f7afb4907a9bb45e1736a3bf4', 'request_id': 'req-0a288277-9bc8-47d3-9d27-5b609d208b2b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:37.574926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] The token cache is being invalidate because project e31b116f7afb4907a9bb45e1736a3bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:37.575216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:37.575853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:37.577423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1118/2231] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:37 2022] DELETE /identity/v3/projects/e31b116f7afb4907a9bb45e1736a3bf4 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:37.581537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640b418-a3fd-450d-8365-24fb91956401 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:37.591143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640b418-a3fd-450d-8365-24fb91956401 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:37.647350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d1a7f-2dd3-47c6-a318-86c697d6a410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:37.648238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d1a7f-2dd3-47c6-a318-86c697d6a410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:37.648521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d1a7f-2dd3-47c6-a318-86c697d6a410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:37.648796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d1a7f-2dd3-47c6-a318-86c697d6a410 admin admin] PATH_INFO: `/v3/users/f802d63d71b14e57b2789917754abb87` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:37.649962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d1a7f-2dd3-47c6-a318-86c697d6a410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f802d63d71b14e57b2789917754abb87)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:37.651957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d1a7f-2dd3-47c6-a318-86c697d6a410 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:37.708300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-119d1a7f-2dd3-47c6-a318-86c697d6a410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f802d63d71b14e57b2789917754abb87'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:37.720618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-119d1a7f-2dd3-47c6-a318-86c697d6a410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f802d63d71b14e57b2789917754abb87'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:37.724700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:37.728467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-119d1a7f-2dd3-47c6-a318-86c697d6a410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f802d63d71b14e57b2789917754abb87'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:37.729688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-119d1a7f-2dd3-47c6-a318-86c697d6a410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f802d63d71b14e57b2789917754abb87'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:37.737517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-119d1a7f-2dd3-47c6-a318-86c697d6a410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f802d63d71b14e57b2789917754abb87'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:37.743422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1114/2232] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:37 2022] DELETE /identity/v3/users/f802d63d71b14e57b2789917754abb87 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:38.095676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:38.097189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:38.110414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a288277-9bc8-47d3-9d27-5b609d208b2b admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:38.123611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:38.169534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:38.170431 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.170712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.170982 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] PATH_INFO: `/v3/projects/61d14dd3157044fb838328e701423f05` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.172130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d14dd3157044fb838328e701423f05)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:38.174048 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:38.219240 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d14dd3157044fb838328e701423f05', 'request_id': 'req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:38.226344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d14dd3157044fb838328e701423f05', 'request_id': 'req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:38.231936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d14dd3157044fb838328e701423f05', 'request_id': 'req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:38.243253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] The token cache is being invalidate because project 61d14dd3157044fb838328e701423f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:02:38.243479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:38.244117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1e14b4b1-a97c-4a6b-bbea-06b3d4d6e994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:38.245891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1119/2233] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:38 2022] DELETE /identity/v3/projects/61d14dd3157044fb838328e701423f05 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:38.250835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daca049-ca34-457d-b0ca-bc3742332292 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.251130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daca049-ca34-457d-b0ca-bc3742332292 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.251414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daca049-ca34-457d-b0ca-bc3742332292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.270361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5daca049-ca34-457d-b0ca-bc3742332292 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:38.271607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5daca049-ca34-457d-b0ca-bc3742332292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:38.331662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1115/2234] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:38.340065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a082b54-fd3a-439d-9795-a72246d46098 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.340364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a082b54-fd3a-439d-9795-a72246d46098 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.340646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a082b54-fd3a-439d-9795-a72246d46098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.359282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6a082b54-fd3a-439d-9795-a72246d46098 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:38.360489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6a082b54-fd3a-439d-9795-a72246d46098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:38.368546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1120/2235] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:38.373299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daca049-ca34-457d-b0ca-bc3742332292 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:38.383728 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daca049-ca34-457d-b0ca-bc3742332292 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:38.389094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d2c95-e70b-4cef-b557-038f1019e848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:38.389860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d2c95-e70b-4cef-b557-038f1019e848 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.390208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d2c95-e70b-4cef-b557-038f1019e848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.390692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d2c95-e70b-4cef-b557-038f1019e848 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.391523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d2c95-e70b-4cef-b557-038f1019e848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:02:38.391895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d2c95-e70b-4cef-b557-038f1019e848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:38.393578 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d2c95-e70b-4cef-b557-038f1019e848 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:38.408454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1116/2236] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:02:38.413439 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a082b54-fd3a-439d-9795-a72246d46098 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:38.429627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a082b54-fd3a-439d-9795-a72246d46098 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:38.443401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d559d-cf93-4dff-b3e0-0d8337311e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:38.444392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d559d-cf93-4dff-b3e0-0d8337311e82 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.444980 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d559d-cf93-4dff-b3e0-0d8337311e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.445415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d559d-cf93-4dff-b3e0-0d8337311e82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.446838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d559d-cf93-4dff-b3e0-0d8337311e82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:38.449668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d559d-cf93-4dff-b3e0-0d8337311e82 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:38.478604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1121/2237] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:38 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:38.482670 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d2c95-e70b-4cef-b557-038f1019e848 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:38.492534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d2c95-e70b-4cef-b557-038f1019e848 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:38.548252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eb6c4-ce28-4c05-9a96-b98793b82c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:38.548849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eb6c4-ce28-4c05-9a96-b98793b82c2c admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.549135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eb6c4-ce28-4c05-9a96-b98793b82c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.549413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eb6c4-ce28-4c05-9a96-b98793b82c2c admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.550127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0eb6c4-ce28-4c05-9a96-b98793b82c2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:38.552106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0eb6c4-ce28-4c05-9a96-b98793b82c2c admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:38.571696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1117/2238] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:38 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:02:38.575700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d559d-cf93-4dff-b3e0-0d8337311e82 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:38.586636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d559d-cf93-4dff-b3e0-0d8337311e82 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:38.591284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7aa601-e2d5-4e6a-9820-163009913e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:38.591943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7aa601-e2d5-4e6a-9820-163009913e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.592226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7aa601-e2d5-4e6a-9820-163009913e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.592503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7aa601-e2d5-4e6a-9820-163009913e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.593251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7aa601-e2d5-4e6a-9820-163009913e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:38.594895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7aa601-e2d5-4e6a-9820-163009913e65 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:38.605098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1122/2239] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:38.608863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eb6c4-ce28-4c05-9a96-b98793b82c2c admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:38.618608 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eb6c4-ce28-4c05-9a96-b98793b82c2c admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:38.623407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f499ee-1737-4722-9f2f-c0a32b87b135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:38.624050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f499ee-1737-4722-9f2f-c0a32b87b135 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.624332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f499ee-1737-4722-9f2f-c0a32b87b135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.624609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f499ee-1737-4722-9f2f-c0a32b87b135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.625348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f499ee-1737-4722-9f2f-c0a32b87b135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:38.626978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f499ee-1737-4722-9f2f-c0a32b87b135 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:38.637106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1118/2240] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:38.640979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7aa601-e2d5-4e6a-9820-163009913e65 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:38.651288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7aa601-e2d5-4e6a-9820-163009913e65 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:38.655804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fabb9f-6a56-4103-a1e4-7128c300e4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:38.656661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fabb9f-6a56-4103-a1e4-7128c300e4bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.656940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fabb9f-6a56-4103-a1e4-7128c300e4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.657208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fabb9f-6a56-4103-a1e4-7128c300e4bd admin admin] PATH_INFO: `/v3/projects/9c47e22804064370a18e83bebdf3a219/users/0e9f1aeab6084fba9d404d108cade29d/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.671170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fabb9f-6a56-4103-a1e4-7128c300e4bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c47e22804064370a18e83bebdf3a219, user_id=0e9f1aeab6084fba9d404d108cade29d, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:38.673845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fabb9f-6a56-4103-a1e4-7128c300e4bd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:38.686426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1123/2241] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:38 2022] PUT /identity/v3/projects/9c47e22804064370a18e83bebdf3a219/users/0e9f1aeab6084fba9d404d108cade29d/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:38.698119 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad22857-d1bc-4562-b7b0-b61b5cc727d3 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.698453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad22857-d1bc-4562-b7b0-b61b5cc727d3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.698743 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad22857-d1bc-4562-b7b0-b61b5cc727d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.725245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-8ad22857-d1bc-4562-b7b0-b61b5cc727d3 None None] MFA Rules not processed for user `0e9f1aeab6084fba9d404d108cade29d`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:38.726518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8ad22857-d1bc-4562-b7b0-b61b5cc727d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:38.766797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:38.791363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1119/2242] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:38.795794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fabb9f-6a56-4103-a1e4-7128c300e4bd admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:38.839898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:38.858542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fabb9f-6a56-4103-a1e4-7128c300e4bd admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:38.863171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc35b4-cd71-4875-9f54-d53d13549453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:38.863821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc35b4-cd71-4875-9f54-d53d13549453 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.864107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc35b4-cd71-4875-9f54-d53d13549453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.864912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc35b4-cd71-4875-9f54-d53d13549453 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.865594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc35b4-cd71-4875-9f54-d53d13549453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:38.867545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc35b4-cd71-4875-9f54-d53d13549453 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:38.892224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1124/2243] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:02:38 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:38.896287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad22857-d1bc-4562-b7b0-b61b5cc727d3 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:38.906825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad22857-d1bc-4562-b7b0-b61b5cc727d3 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:38.962990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e639c9c-6121-4ede-9737-8fa4ecc5d649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:38.963674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e639c9c-6121-4ede-9737-8fa4ecc5d649 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:38.963957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e639c9c-6121-4ede-9737-8fa4ecc5d649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:38.964235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e639c9c-6121-4ede-9737-8fa4ecc5d649 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:38.964926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e639c9c-6121-4ede-9737-8fa4ecc5d649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:38.966890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e639c9c-6121-4ede-9737-8fa4ecc5d649 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:38.990884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1120/2244] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:02:38 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:02:38.995112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc35b4-cd71-4875-9f54-d53d13549453 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:39.005713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc35b4-cd71-4875-9f54-d53d13549453 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:39.011011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1390329-1de8-4946-b559-c2fd253a7929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:39.011669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1390329-1de8-4946-b559-c2fd253a7929 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:39.011953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1390329-1de8-4946-b559-c2fd253a7929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:39.012286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1390329-1de8-4946-b559-c2fd253a7929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:39.013032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1390329-1de8-4946-b559-c2fd253a7929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:39.014670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1390329-1de8-4946-b559-c2fd253a7929 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:39.025513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1125/2245] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:39.029459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e639c9c-6121-4ede-9737-8fa4ecc5d649 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:39.039574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e639c9c-6121-4ede-9737-8fa4ecc5d649 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:39.044441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e385c-62bf-4d60-aef7-e717157d1d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:39.045316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e385c-62bf-4d60-aef7-e717157d1d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:39.045601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e385c-62bf-4d60-aef7-e717157d1d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:39.045909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e385c-62bf-4d60-aef7-e717157d1d49 admin admin] PATH_INFO: `/v3/projects/fa106ca377a442a79a3239cbe07ca444/users/7b4ecd708da64439882ec41c0ad6cb28/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:39.060723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e385c-62bf-4d60-aef7-e717157d1d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa106ca377a442a79a3239cbe07ca444, user_id=7b4ecd708da64439882ec41c0ad6cb28, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:39.063407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e385c-62bf-4d60-aef7-e717157d1d49 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:39.076326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1121/2246] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:39 2022] PUT /identity/v3/projects/fa106ca377a442a79a3239cbe07ca444/users/7b4ecd708da64439882ec41c0ad6cb28/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:02:39.080358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1390329-1de8-4946-b559-c2fd253a7929 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:39.090374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1390329-1de8-4946-b559-c2fd253a7929 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:39.147110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a2235-6220-4b59-bc75-96451b2d57ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:39.147819 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a2235-6220-4b59-bc75-96451b2d57ac admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:39.148235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a2235-6220-4b59-bc75-96451b2d57ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:39.148519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a2235-6220-4b59-bc75-96451b2d57ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:39.149243 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a2235-6220-4b59-bc75-96451b2d57ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:39.151970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a2235-6220-4b59-bc75-96451b2d57ac admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:39.165944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1126/2247] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:02:39.166975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e385c-62bf-4d60-aef7-e717157d1d49 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:39.177290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e385c-62bf-4d60-aef7-e717157d1d49 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:39.182618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad290ff-ad37-4ebc-8656-647b786d1f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:39.183504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad290ff-ad37-4ebc-8656-647b786d1f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:39.183786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad290ff-ad37-4ebc-8656-647b786d1f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:39.184061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad290ff-ad37-4ebc-8656-647b786d1f30 admin admin] PATH_INFO: `/v3/projects/fa106ca377a442a79a3239cbe07ca444/users/7b4ecd708da64439882ec41c0ad6cb28/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:39.186049 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad290ff-ad37-4ebc-8656-647b786d1f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa106ca377a442a79a3239cbe07ca444, user_id=7b4ecd708da64439882ec41c0ad6cb28, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:39.188749 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad290ff-ad37-4ebc-8656-647b786d1f30 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:39.199769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1122/2248] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:02:39 2022] PUT /identity/v3/projects/fa106ca377a442a79a3239cbe07ca444/users/7b4ecd708da64439882ec41c0ad6cb28/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:39.209794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f2b78-0668-4c23-8bbb-6f5ea057c08b None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:39.210084 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f2b78-0668-4c23-8bbb-6f5ea057c08b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:39.210400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f2b78-0668-4c23-8bbb-6f5ea057c08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:39.236837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-de8f2b78-0668-4c23-8bbb-6f5ea057c08b None None] MFA Rules not processed for user `7b4ecd708da64439882ec41c0ad6cb28`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:39.238049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-de8f2b78-0668-4c23-8bbb-6f5ea057c08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:39.308097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1127/2249] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:39.318688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad290ff-ad37-4ebc-8656-647b786d1f30 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:39.320373 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9ad290ff-ad37-4ebc-8656-647b786d1f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:39.376281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad290ff-ad37-4ebc-8656-647b786d1f30 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:39.379856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ce866-336b-4304-9bc5-419e452a3ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:39.381415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ce866-336b-4304-9bc5-419e452a3ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:39.381415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ce866-336b-4304-9bc5-419e452a3ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:39.381612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ce866-336b-4304-9bc5-419e452a3ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:39.391043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ce866-336b-4304-9bc5-419e452a3ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:39.392501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ce866-336b-4304-9bc5-419e452a3ad9 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:39.405583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1123/2250] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:39.504808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f2b78-0668-4c23-8bbb-6f5ea057c08b None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:39.515970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f2b78-0668-4c23-8bbb-6f5ea057c08b None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:39.519952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62fb36-5282-4148-bb6f-7aa5b4099d23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:39.521214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62fb36-5282-4148-bb6f-7aa5b4099d23 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:39.521568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62fb36-5282-4148-bb6f-7aa5b4099d23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:39.521912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62fb36-5282-4148-bb6f-7aa5b4099d23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:39.557950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:39.582425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62fb36-5282-4148-bb6f-7aa5b4099d23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:39.584393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62fb36-5282-4148-bb6f-7aa5b4099d23 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:39.593439 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1128/2251] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:49.399924 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:49.589265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=11 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:57.390128 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986b90a-e6d3-4c21-9415-1364a022094f None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:57.390603 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986b90a-e6d3-4c21-9415-1364a022094f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:57.390747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986b90a-e6d3-4c21-9415-1364a022094f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:57.451187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-4986b90a-e6d3-4c21-9415-1364a022094f None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:57.452377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4986b90a-e6d3-4c21-9415-1364a022094f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:57.519399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1124/2252] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 13:02:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:57.563394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62fb36-5282-4148-bb6f-7aa5b4099d23 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:57.564937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ef62fb36-5282-4148-bb6f-7aa5b4099d23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:57.669635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62fb36-5282-4148-bb6f-7aa5b4099d23 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:57.676288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64621bb7-54af-48b7-91c8-29eaab3d9611 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:57.677041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64621bb7-54af-48b7-91c8-29eaab3d9611 service swift] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:57.677501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64621bb7-54af-48b7-91c8-29eaab3d9611 service swift] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:57.677784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64621bb7-54af-48b7-91c8-29eaab3d9611 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:57.687274 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64621bb7-54af-48b7-91c8-29eaab3d9611 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:57.688926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64621bb7-54af-48b7-91c8-29eaab3d9611 service swift] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:57.697099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1129/2253] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:02:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:02:59.004202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986b90a-e6d3-4c21-9415-1364a022094f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:59.113248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986b90a-e6d3-4c21-9415-1364a022094f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:59.123285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:59.124380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:59.125710 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:59.126773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] PATH_INFO: `/v3/users/0e9f1aeab6084fba9d404d108cade29d` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:59.129056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e9f1aeab6084fba9d404d108cade29d)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:59.133246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:59.204016 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e9f1aeab6084fba9d404d108cade29d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:59.243655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e9f1aeab6084fba9d404d108cade29d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:59.245649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:02:59.256333 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e9f1aeab6084fba9d404d108cade29d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:59.259375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e9f1aeab6084fba9d404d108cade29d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:59.272146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e9f1aeab6084fba9d404d108cade29d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:02:59.281109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1125/2254] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:02:59 2022] DELETE /identity/v3/users/0e9f1aeab6084fba9d404d108cade29d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:02:59.286524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f46b1-a940-4466-bfbc-382d5a0879b5 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:59.287105 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f46b1-a940-4466-bfbc-382d5a0879b5 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:59.287404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f46b1-a940-4466-bfbc-382d5a0879b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:59.315104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-786f46b1-a940-4466-bfbc-382d5a0879b5 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:02:59.316686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-786f46b1-a940-4466-bfbc-382d5a0879b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:59.439550 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1130/2255] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:02:59.449765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:59.451796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:59.524022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb562cc-a6ca-4ad8-88d4-2d408282dc87 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:59.528960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4d64f-ef17-4d2e-9e5f-60a3cd6d95ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:59.529630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d64f-ef17-4d2e-9e5f-60a3cd6d95ac service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:59.529917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d64f-ef17-4d2e-9e5f-60a3cd6d95ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:59.530198 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d64f-ef17-4d2e-9e5f-60a3cd6d95ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:59.540798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4d64f-ef17-4d2e-9e5f-60a3cd6d95ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:59.542323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4d64f-ef17-4d2e-9e5f-60a3cd6d95ac service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:59.552764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1126/2256] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:02:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:02:59.874054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f46b1-a940-4466-bfbc-382d5a0879b5 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:02:59.878221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-786f46b1-a940-4466-bfbc-382d5a0879b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:02:59.895696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f46b1-a940-4466-bfbc-382d5a0879b5 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:02:59.917981 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:02:59.918842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:02:59.919124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:02:59.919395 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] PATH_INFO: `/v3/projects/9c47e22804064370a18e83bebdf3a219` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:02:59.921702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c47e22804064370a18e83bebdf3a219)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:02:59.927611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:02:59.998213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c47e22804064370a18e83bebdf3a219', 'request_id': 'req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.007767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c47e22804064370a18e83bebdf3a219', 'request_id': 'req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.015734 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c47e22804064370a18e83bebdf3a219', 'request_id': 'req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.017377 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:00.041638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] The token cache is being invalidate because project 9c47e22804064370a18e83bebdf3a219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:03:00.041968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.042733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.044985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1131/2257] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:02:59 2022] DELETE /identity/v3/projects/9c47e22804064370a18e83bebdf3a219 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:00.048949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4d64f-ef17-4d2e-9e5f-60a3cd6d95ac service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:00.060224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4d64f-ef17-4d2e-9e5f-60a3cd6d95ac service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:00.126901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ff39f-7b36-4d58-a45a-7de9b24dd1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:00.127862 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ff39f-7b36-4d58-a45a-7de9b24dd1ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:00.128146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ff39f-7b36-4d58-a45a-7de9b24dd1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:00.128513 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ff39f-7b36-4d58-a45a-7de9b24dd1ae admin admin] PATH_INFO: `/v3/users/7b4ecd708da64439882ec41c0ad6cb28` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:00.130212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ff39f-7b36-4d58-a45a-7de9b24dd1ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b4ecd708da64439882ec41c0ad6cb28)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:00.132317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ff39f-7b36-4d58-a45a-7de9b24dd1ae admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:00.219453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d18ff39f-7b36-4d58-a45a-7de9b24dd1ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b4ecd708da64439882ec41c0ad6cb28'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.234988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d18ff39f-7b36-4d58-a45a-7de9b24dd1ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b4ecd708da64439882ec41c0ad6cb28'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.259566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d18ff39f-7b36-4d58-a45a-7de9b24dd1ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b4ecd708da64439882ec41c0ad6cb28'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.260775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d18ff39f-7b36-4d58-a45a-7de9b24dd1ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b4ecd708da64439882ec41c0ad6cb28'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.272404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d18ff39f-7b36-4d58-a45a-7de9b24dd1ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b4ecd708da64439882ec41c0ad6cb28'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.279128 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1127/2258] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:03:00 2022] DELETE /identity/v3/users/7b4ecd708da64439882ec41c0ad6cb28 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:00.673992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:00.675666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:00.699499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6ccc0-76c2-4e7c-bb70-c1c6692ea621 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:00.799158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:00.800064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:00.800352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:00.800628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] PATH_INFO: `/v3/projects/fa106ca377a442a79a3239cbe07ca444` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:00.801980 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa106ca377a442a79a3239cbe07ca444)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:00.804311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:00.865631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa106ca377a442a79a3239cbe07ca444', 'request_id': 'req-b8148063-9ec7-4c65-b5d8-ec6b9a848957'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.874437 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa106ca377a442a79a3239cbe07ca444', 'request_id': 'req-b8148063-9ec7-4c65-b5d8-ec6b9a848957'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.882074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa106ca377a442a79a3239cbe07ca444', 'request_id': 'req-b8148063-9ec7-4c65-b5d8-ec6b9a848957'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.895358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] The token cache is being invalidate because project fa106ca377a442a79a3239cbe07ca444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:03:00.895730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.896478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b8148063-9ec7-4c65-b5d8-ec6b9a848957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:00.898674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1132/2259] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:03:00 2022] DELETE /identity/v3/projects/fa106ca377a442a79a3239cbe07ca444 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:00.903864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201036d-ab04-4dd6-8884-9d01a473ee79 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:00.904160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201036d-ab04-4dd6-8884-9d01a473ee79 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:00.904438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201036d-ab04-4dd6-8884-9d01a473ee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:00.930071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:00.956479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9201036d-ab04-4dd6-8884-9d01a473ee79 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:00.960096 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9201036d-ab04-4dd6-8884-9d01a473ee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:01.057797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1128/2260] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:03:01.068617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0715b-0382-4802-9895-2cb85ec816cb None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:01.068900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0715b-0382-4802-9895-2cb85ec816cb None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:01.069225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0715b-0382-4802-9895-2cb85ec816cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:01.090977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-1bc0715b-0382-4802-9895-2cb85ec816cb None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:01.092231 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1bc0715b-0382-4802-9895-2cb85ec816cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:01.105383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1133/2261] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:01.108754 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201036d-ab04-4dd6-8884-9d01a473ee79 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:01.120493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201036d-ab04-4dd6-8884-9d01a473ee79 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:01.125185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb554013-72b3-43c6-aa79-f6c7a15ee0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:01.125982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb554013-72b3-43c6-aa79-f6c7a15ee0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:01.126273 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb554013-72b3-43c6-aa79-f6c7a15ee0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:01.126574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb554013-72b3-43c6-aa79-f6c7a15ee0d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:01.127305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb554013-72b3-43c6-aa79-f6c7a15ee0d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:03:01.127614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb554013-72b3-43c6-aa79-f6c7a15ee0d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:01.129933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb554013-72b3-43c6-aa79-f6c7a15ee0d9 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:01.144322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1129/2262] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:03:01.148912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0715b-0382-4802-9895-2cb85ec816cb None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:01.159015 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0715b-0382-4802-9895-2cb85ec816cb None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:01.163783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e603a86-22d6-4e8d-b6df-6656104925f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:01.164507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e603a86-22d6-4e8d-b6df-6656104925f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:01.164796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e603a86-22d6-4e8d-b6df-6656104925f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:01.165077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e603a86-22d6-4e8d-b6df-6656104925f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:01.165777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e603a86-22d6-4e8d-b6df-6656104925f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:01.167735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e603a86-22d6-4e8d-b6df-6656104925f3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:01.208421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1134/2263] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:03:01 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:03:01.209854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb554013-72b3-43c6-aa79-f6c7a15ee0d9 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:01.224470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb554013-72b3-43c6-aa79-f6c7a15ee0d9 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:01.308031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7f576-5e16-4159-a8a4-0bfccfde1dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:01.308733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f576-5e16-4159-a8a4-0bfccfde1dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:01.309024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f576-5e16-4159-a8a4-0bfccfde1dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:01.309306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f576-5e16-4159-a8a4-0bfccfde1dd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:01.310039 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7f576-5e16-4159-a8a4-0bfccfde1dd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:01.312206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7f576-5e16-4159-a8a4-0bfccfde1dd1 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:01.337040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1130/2264] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:03:01 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:03:01.341370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e603a86-22d6-4e8d-b6df-6656104925f3 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:01.355539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e603a86-22d6-4e8d-b6df-6656104925f3 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:01.360846 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b930091-f17e-4638-88fd-6df9c0670df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:01.361519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b930091-f17e-4638-88fd-6df9c0670df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:01.361806 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b930091-f17e-4638-88fd-6df9c0670df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:01.362088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b930091-f17e-4638-88fd-6df9c0670df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:01.362858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b930091-f17e-4638-88fd-6df9c0670df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:01.365056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b930091-f17e-4638-88fd-6df9c0670df8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:01.377148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1135/2265] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:03:01.381240 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7f576-5e16-4159-a8a4-0bfccfde1dd1 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:01.393258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7f576-5e16-4159-a8a4-0bfccfde1dd1 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:01.409336 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0669d-d077-4d0e-90b8-6f82d5898533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:01.410354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0669d-d077-4d0e-90b8-6f82d5898533 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:01.411389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0669d-d077-4d0e-90b8-6f82d5898533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:01.411684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0669d-d077-4d0e-90b8-6f82d5898533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:01.412479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0669d-d077-4d0e-90b8-6f82d5898533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:01.414426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0669d-d077-4d0e-90b8-6f82d5898533 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:01.425648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1131/2266] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:03:01.429726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b930091-f17e-4638-88fd-6df9c0670df8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:01.440519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b930091-f17e-4638-88fd-6df9c0670df8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:01.445654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fda98-139b-4e3a-8334-5a6253fde639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:01.446586 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fda98-139b-4e3a-8334-5a6253fde639 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:01.446873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fda98-139b-4e3a-8334-5a6253fde639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:01.447149 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fda98-139b-4e3a-8334-5a6253fde639 admin admin] PATH_INFO: `/v3/projects/4102dd367da141398b035bf51b92071d/users/e0ed31ab86c34fe2bae54f1dfe3c4fb7/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:01.481635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644fda98-139b-4e3a-8334-5a6253fde639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4102dd367da141398b035bf51b92071d, user_id=e0ed31ab86c34fe2bae54f1dfe3c4fb7, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:01.484573 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644fda98-139b-4e3a-8334-5a6253fde639 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:01.511188 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1136/2267] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:03:01 2022] PUT /identity/v3/projects/4102dd367da141398b035bf51b92071d/users/e0ed31ab86c34fe2bae54f1dfe3c4fb7/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:01.522337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a4423-8b1a-477f-8ac3-c76eca9f83cc None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:01.522337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a4423-8b1a-477f-8ac3-c76eca9f83cc None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:01.522337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a4423-8b1a-477f-8ac3-c76eca9f83cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:01.562568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-152a4423-8b1a-477f-8ac3-c76eca9f83cc None None] MFA Rules not processed for user `e0ed31ab86c34fe2bae54f1dfe3c4fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:01.566397 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-152a4423-8b1a-477f-8ac3-c76eca9f83cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:01.653936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:01.660149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1132/2268] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:01.666559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fda98-139b-4e3a-8334-5a6253fde639 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:01.696964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:01.800952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fda98-139b-4e3a-8334-5a6253fde639 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:01.810141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e137c-0b03-4caf-84e8-97f85bd07c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:01.811388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e137c-0b03-4caf-84e8-97f85bd07c20 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:01.812237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e137c-0b03-4caf-84e8-97f85bd07c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:01.812633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e137c-0b03-4caf-84e8-97f85bd07c20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:01.813844 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e137c-0b03-4caf-84e8-97f85bd07c20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:01.817380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e137c-0b03-4caf-84e8-97f85bd07c20 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:01.856053 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1137/2269] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:03:01 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:03:01.860948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a4423-8b1a-477f-8ac3-c76eca9f83cc None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:01.874215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a4423-8b1a-477f-8ac3-c76eca9f83cc None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:01.960984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c40e1-bb14-4ca8-9469-da1259523dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:01.962006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c40e1-bb14-4ca8-9469-da1259523dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:01.962309 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c40e1-bb14-4ca8-9469-da1259523dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:01.962681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c40e1-bb14-4ca8-9469-da1259523dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:01.964350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c40e1-bb14-4ca8-9469-da1259523dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:01.965755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c40e1-bb14-4ca8-9469-da1259523dd2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:01.998464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1133/2270] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:03:01 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:03:01.999538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e137c-0b03-4caf-84e8-97f85bd07c20 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:02.013135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e137c-0b03-4caf-84e8-97f85bd07c20 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:02.021302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c235ef-5581-43f6-bcbf-ccdd67a41849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:02.021982 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c235ef-5581-43f6-bcbf-ccdd67a41849 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:02.022266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c235ef-5581-43f6-bcbf-ccdd67a41849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:02.022644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c235ef-5581-43f6-bcbf-ccdd67a41849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:02.023338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c235ef-5581-43f6-bcbf-ccdd67a41849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:02.025041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c235ef-5581-43f6-bcbf-ccdd67a41849 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:02.037675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1138/2271] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:03:02.041700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c40e1-bb14-4ca8-9469-da1259523dd2 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:02.053080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c40e1-bb14-4ca8-9469-da1259523dd2 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:02.057898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d5813-7cd1-4867-9fc4-df9855948341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:02.058610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5813-7cd1-4867-9fc4-df9855948341 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:02.058898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5813-7cd1-4867-9fc4-df9855948341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:02.059179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5813-7cd1-4867-9fc4-df9855948341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:02.061367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d5813-7cd1-4867-9fc4-df9855948341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:02.062318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d5813-7cd1-4867-9fc4-df9855948341 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:02.073612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1134/2272] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:03:02.077772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c235ef-5581-43f6-bcbf-ccdd67a41849 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:02.089022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c235ef-5581-43f6-bcbf-ccdd67a41849 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:02.095768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732259df-43c0-46dd-86e8-0db4f88661a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:02.096675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732259df-43c0-46dd-86e8-0db4f88661a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:02.096961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732259df-43c0-46dd-86e8-0db4f88661a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:02.097233 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732259df-43c0-46dd-86e8-0db4f88661a1 admin admin] PATH_INFO: `/v3/projects/6876e57af831480cad702138d7ee77df/users/98f4486a1b824c369be44602558cb0d0/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:02.122814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732259df-43c0-46dd-86e8-0db4f88661a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6876e57af831480cad702138d7ee77df, user_id=98f4486a1b824c369be44602558cb0d0, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:02.125511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732259df-43c0-46dd-86e8-0db4f88661a1 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:02.141145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1139/2273] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:03:02 2022] PUT /identity/v3/projects/6876e57af831480cad702138d7ee77df/users/98f4486a1b824c369be44602558cb0d0/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:02.152200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c1996-be77-4896-8fc6-18ddc636bde5 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:02.152488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c1996-be77-4896-8fc6-18ddc636bde5 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:02.152784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c1996-be77-4896-8fc6-18ddc636bde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:02.182808 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-3d5c1996-be77-4896-8fc6-18ddc636bde5 None None] MFA Rules not processed for user `98f4486a1b824c369be44602558cb0d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:02.184085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3d5c1996-be77-4896-8fc6-18ddc636bde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:02.257684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1135/2274] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:02.262236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732259df-43c0-46dd-86e8-0db4f88661a1 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:02.342609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732259df-43c0-46dd-86e8-0db4f88661a1 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:02.350401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0278d-aeaf-4097-839c-691eec378522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:02.350401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0278d-aeaf-4097-839c-691eec378522 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:02.350868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0278d-aeaf-4097-839c-691eec378522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:02.350868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0278d-aeaf-4097-839c-691eec378522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:02.351878 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0278d-aeaf-4097-839c-691eec378522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:02.353684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0278d-aeaf-4097-839c-691eec378522 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:02.390421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1140/2275] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:03:02 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:03:02.394513 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c1996-be77-4896-8fc6-18ddc636bde5 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:02.416408 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c1996-be77-4896-8fc6-18ddc636bde5 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:02.491442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0724a-8e92-4e37-a3f7-8559961cd64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:02.492157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0724a-8e92-4e37-a3f7-8559961cd64f admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:02.492443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0724a-8e92-4e37-a3f7-8559961cd64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:02.492716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0724a-8e92-4e37-a3f7-8559961cd64f admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:02.493414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0724a-8e92-4e37-a3f7-8559961cd64f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:02.495738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0724a-8e92-4e37-a3f7-8559961cd64f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:02.512523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:02.531271 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1136/2276] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:03:02 2022] POST /identity/v3/users => generated 298 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:03:02.533343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0278d-aeaf-4097-839c-691eec378522 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:02.547850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0278d-aeaf-4097-839c-691eec378522 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:02.553183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06091434-4cfc-4ad6-b68d-a71ed9c76ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:02.553870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091434-4cfc-4ad6-b68d-a71ed9c76ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:02.554157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091434-4cfc-4ad6-b68d-a71ed9c76ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:02.554471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091434-4cfc-4ad6-b68d-a71ed9c76ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:02.555290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06091434-4cfc-4ad6-b68d-a71ed9c76ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:02.556916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06091434-4cfc-4ad6-b68d-a71ed9c76ad3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:02.570933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1141/2277] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:03:02.575145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0724a-8e92-4e37-a3f7-8559961cd64f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:02.586382 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0724a-8e92-4e37-a3f7-8559961cd64f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:02.591266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2d5f1-5101-4751-b187-b50c7a69afbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:02.592434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2d5f1-5101-4751-b187-b50c7a69afbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:02.592825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2d5f1-5101-4751-b187-b50c7a69afbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:02.593225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2d5f1-5101-4751-b187-b50c7a69afbd admin admin] PATH_INFO: `/v3/projects/c325d6f3e99d4bb48790d23278a5986b/users/5d0a8d0ac7f6493baf848583a9f58d07/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:02.610785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2d5f1-5101-4751-b187-b50c7a69afbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c325d6f3e99d4bb48790d23278a5986b, user_id=5d0a8d0ac7f6493baf848583a9f58d07, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:02.613751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2d5f1-5101-4751-b187-b50c7a69afbd admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:02.630649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1137/2278] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:03:02 2022] PUT /identity/v3/projects/c325d6f3e99d4bb48790d23278a5986b/users/5d0a8d0ac7f6493baf848583a9f58d07/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:03:02.634428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06091434-4cfc-4ad6-b68d-a71ed9c76ad3 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:02.644803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06091434-4cfc-4ad6-b68d-a71ed9c76ad3 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:02.689953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:02.708388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc4305-10e2-48e7-91a6-c18903a9b5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:02.709059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc4305-10e2-48e7-91a6-c18903a9b5da admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:02.709336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc4305-10e2-48e7-91a6-c18903a9b5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:02.709618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc4305-10e2-48e7-91a6-c18903a9b5da admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:02.710378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc4305-10e2-48e7-91a6-c18903a9b5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:02.712023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc4305-10e2-48e7-91a6-c18903a9b5da admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:02.726799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1142/2279] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:03:02.728518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2d5f1-5101-4751-b187-b50c7a69afbd admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:02.741024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2d5f1-5101-4751-b187-b50c7a69afbd admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:02.746972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc097f-547d-4837-bdb2-c333ce241013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:02.747954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc097f-547d-4837-bdb2-c333ce241013 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:02.748308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc097f-547d-4837-bdb2-c333ce241013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:02.748652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc097f-547d-4837-bdb2-c333ce241013 admin admin] PATH_INFO: `/v3/projects/c325d6f3e99d4bb48790d23278a5986b/users/5d0a8d0ac7f6493baf848583a9f58d07/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:02.750945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc097f-547d-4837-bdb2-c333ce241013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c325d6f3e99d4bb48790d23278a5986b, user_id=5d0a8d0ac7f6493baf848583a9f58d07, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:02.753742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc097f-547d-4837-bdb2-c333ce241013 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:02.780204 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1138/2280] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:03:02 2022] PUT /identity/v3/projects/c325d6f3e99d4bb48790d23278a5986b/users/5d0a8d0ac7f6493baf848583a9f58d07/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:02.781356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ce2bb-5c26-4275-89f2-a218c214ffd6 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:02.781754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ce2bb-5c26-4275-89f2-a218c214ffd6 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:02.782122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ce2bb-5c26-4275-89f2-a218c214ffd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:02.815575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b61ce2bb-5c26-4275-89f2-a218c214ffd6 None None] MFA Rules not processed for user `5d0a8d0ac7f6493baf848583a9f58d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:02.817381 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b61ce2bb-5c26-4275-89f2-a218c214ffd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:02.902658 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1143/2281] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:02.915287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc097f-547d-4837-bdb2-c333ce241013 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:02.918067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-60dc097f-547d-4837-bdb2-c333ce241013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:03.009557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc097f-547d-4837-bdb2-c333ce241013 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:03.018900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a0f78-65ef-4c01-a74b-fae08f16b63c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:03.020378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0f78-65ef-4c01-a74b-fae08f16b63c service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:03.021169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0f78-65ef-4c01-a74b-fae08f16b63c service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:03.021462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0f78-65ef-4c01-a74b-fae08f16b63c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:03.101858 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a0f78-65ef-4c01-a74b-fae08f16b63c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:03.104365 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a0f78-65ef-4c01-a74b-fae08f16b63c service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:03.135965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1139/2282] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:03:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:03:03.311773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ce2bb-5c26-4275-89f2-a218c214ffd6 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:03.313509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b61ce2bb-5c26-4275-89f2-a218c214ffd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:03.498558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ce2bb-5c26-4275-89f2-a218c214ffd6 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:03.509302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219943c-cb36-4d0e-ba68-cea8e59ea228 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:03.510171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219943c-cb36-4d0e-ba68-cea8e59ea228 service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:03.511217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219943c-cb36-4d0e-ba68-cea8e59ea228 service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:03.511540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219943c-cb36-4d0e-ba68-cea8e59ea228 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:03.515198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6219943c-cb36-4d0e-ba68-cea8e59ea228 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:03.548764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219943c-cb36-4d0e-ba68-cea8e59ea228 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:03.550616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219943c-cb36-4d0e-ba68-cea8e59ea228 service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:03.569744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1144/2283] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:03:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:03:04.717948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a0f78-65ef-4c01-a74b-fae08f16b63c service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:04.719679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6a5a0f78-65ef-4c01-a74b-fae08f16b63c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:04.842066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a0f78-65ef-4c01-a74b-fae08f16b63c service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:04.848925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35859833-0dc1-4ac8-b181-6c18534676ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:04.850315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35859833-0dc1-4ac8-b181-6c18534676ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:04.850902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35859833-0dc1-4ac8-b181-6c18534676ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:04.851332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35859833-0dc1-4ac8-b181-6c18534676ec admin admin] PATH_INFO: `/v3/users/a070766ef74348d5a7f353ed53bf5f3e` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:04.853117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35859833-0dc1-4ac8-b181-6c18534676ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=a070766ef74348d5a7f353ed53bf5f3e)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:04.855472 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35859833-0dc1-4ac8-b181-6c18534676ec admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:04.889674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:04.938678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219943c-cb36-4d0e-ba68-cea8e59ea228 service placement] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:04.942407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-35859833-0dc1-4ac8-b181-6c18534676ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a070766ef74348d5a7f353ed53bf5f3e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:04.948934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219943c-cb36-4d0e-ba68-cea8e59ea228 service placement] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:04.954759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f425a-3a9a-4ce8-900a-3e19d4ae1c39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:04.955550 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f425a-3a9a-4ce8-900a-3e19d4ae1c39 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:04.955938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f425a-3a9a-4ce8-900a-3e19d4ae1c39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:04.956493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f425a-3a9a-4ce8-900a-3e19d4ae1c39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:04.959327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-35859833-0dc1-4ac8-b181-6c18534676ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a070766ef74348d5a7f353ed53bf5f3e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:04.970192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-35859833-0dc1-4ac8-b181-6c18534676ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a070766ef74348d5a7f353ed53bf5f3e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:04.971329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-35859833-0dc1-4ac8-b181-6c18534676ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a070766ef74348d5a7f353ed53bf5f3e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:04.980205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-35859833-0dc1-4ac8-b181-6c18534676ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a070766ef74348d5a7f353ed53bf5f3e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:04.987406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1140/2284] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:03:04 2022] DELETE /identity/v3/users/a070766ef74348d5a7f353ed53bf5f3e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:04.996250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92db561-fd09-4bde-8934-7715f1a98311 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:04.996858 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92db561-fd09-4bde-8934-7715f1a98311 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:04.997213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92db561-fd09-4bde-8934-7715f1a98311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:05.012403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:05.023290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a92db561-fd09-4bde-8934-7715f1a98311 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:05.025393 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a92db561-fd09-4bde-8934-7715f1a98311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:05.052304 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f425a-3a9a-4ce8-900a-3e19d4ae1c39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:05.054860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f425a-3a9a-4ce8-900a-3e19d4ae1c39 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:05.085667 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1145/2285] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:03:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:03:05.105766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f425a-3a9a-4ce8-900a-3e19d4ae1c39 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:05.107724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-890f425a-3a9a-4ce8-900a-3e19d4ae1c39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:05.128193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f425a-3a9a-4ce8-900a-3e19d4ae1c39 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:05.174988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1141/2286] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:05.180149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92db561-fd09-4bde-8934-7715f1a98311 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:05.182766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a92db561-fd09-4bde-8934-7715f1a98311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:05.276349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:05.278564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:05.278978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:05.279355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] PATH_INFO: `/v3/users/29c019e09e664381b5f62f01308f1010` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:05.281989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29c019e09e664381b5f62f01308f1010)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:05.286839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:05.358800 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92db561-fd09-4bde-8934-7715f1a98311 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:05.365932 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474732d-97ab-41e1-806d-5835650e982b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:05.367064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474732d-97ab-41e1-806d-5835650e982b service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:05.367475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474732d-97ab-41e1-806d-5835650e982b service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:05.368509 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474732d-97ab-41e1-806d-5835650e982b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:05.408068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29c019e09e664381b5f62f01308f1010'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:05.414050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474732d-97ab-41e1-806d-5835650e982b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:05.415763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474732d-97ab-41e1-806d-5835650e982b service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:05.425897 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29c019e09e664381b5f62f01308f1010'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:05.427838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29c019e09e664381b5f62f01308f1010'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:05.428628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29c019e09e664381b5f62f01308f1010'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:05.443801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1142/2287] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:03:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:03:05.451808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29c019e09e664381b5f62f01308f1010'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:05.469417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1146/2288] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:03:05 2022] DELETE /identity/v3/users/29c019e09e664381b5f62f01308f1010 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:03:05.471933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc0a12-d67d-4aca-ab0b-fb260100e730 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:05.472342 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc0a12-d67d-4aca-ab0b-fb260100e730 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:05.473261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc0a12-d67d-4aca-ab0b-fb260100e730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:05.512809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-68dc0a12-d67d-4aca-ab0b-fb260100e730 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:05.514476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-68dc0a12-d67d-4aca-ab0b-fb260100e730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:05.601446 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:05.657694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1143/2289] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:05.665792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:05.669059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:05.827549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e27472-933e-4677-a45e-545d0b3c5f16 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:05.848707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21591fa-a347-4e96-a0a1-71fa894c6699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:05.849503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21591fa-a347-4e96-a0a1-71fa894c6699 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:05.849686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21591fa-a347-4e96-a0a1-71fa894c6699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:05.849963 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21591fa-a347-4e96-a0a1-71fa894c6699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:05.858864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:05.865040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21591fa-a347-4e96-a0a1-71fa894c6699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:05.868715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21591fa-a347-4e96-a0a1-71fa894c6699 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:05.890555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1147/2290] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:03:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:03:06.125857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc0a12-d67d-4aca-ab0b-fb260100e730 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:06.128215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-68dc0a12-d67d-4aca-ab0b-fb260100e730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:06.260554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc0a12-d67d-4aca-ab0b-fb260100e730 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:06.265252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed8e9a-7027-485b-8c2c-b95153e48860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:06.266122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed8e9a-7027-485b-8c2c-b95153e48860 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:06.266502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed8e9a-7027-485b-8c2c-b95153e48860 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:06.266998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed8e9a-7027-485b-8c2c-b95153e48860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:06.269516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-40ed8e9a-7027-485b-8c2c-b95153e48860 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:06.344977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21591fa-a347-4e96-a0a1-71fa894c6699 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:06.346928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c21591fa-a347-4e96-a0a1-71fa894c6699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:06.368262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21591fa-a347-4e96-a0a1-71fa894c6699 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:06.382737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:06.383612 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:06.383892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:06.384168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] PATH_INFO: `/v3/projects/603e569b8c654f569672037ef13391f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:06.385857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=603e569b8c654f569672037ef13391f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:06.389058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:06.406497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed8e9a-7027-485b-8c2c-b95153e48860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:06.407961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed8e9a-7027-485b-8c2c-b95153e48860 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:06.433458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1144/2291] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:03:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:03:06.516981 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '603e569b8c654f569672037ef13391f6', 'request_id': 'req-aea1a7e3-5edf-45b4-bbdb-3fb149643829'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:06.528914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '603e569b8c654f569672037ef13391f6', 'request_id': 'req-aea1a7e3-5edf-45b4-bbdb-3fb149643829'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:06.542567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '603e569b8c654f569672037ef13391f6', 'request_id': 'req-aea1a7e3-5edf-45b4-bbdb-3fb149643829'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:06.570791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] The token cache is being invalidate because project 603e569b8c654f569672037ef13391f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:03:06.571087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:06.571768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:06.573348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1148/2292] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:03:06 2022] DELETE /identity/v3/projects/603e569b8c654f569672037ef13391f6 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:03:06.583989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ff018-1f37-4f1b-8c01-12c52ea4752d None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:06.584210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ff018-1f37-4f1b-8c01-12c52ea4752d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:06.584453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ff018-1f37-4f1b-8c01-12c52ea4752d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:06.610601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-ce1ff018-1f37-4f1b-8c01-12c52ea4752d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:06.611972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ce1ff018-1f37-4f1b-8c01-12c52ea4752d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:06.643116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:06.646499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:06.734622 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1145/2293] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:06.737223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a9708-e400-4cd0-8c10-141fbdaf7fda None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:06.738850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a9708-e400-4cd0-8c10-141fbdaf7fda None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:06.739817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a9708-e400-4cd0-8c10-141fbdaf7fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:06.755229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1a7e3-5edf-45b4-bbdb-3fb149643829 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:06.760421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:06.761441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:06.761730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:06.762010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] PATH_INFO: `/v3/projects/97d4002620de4768b87ebf323f7cbb43` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:06.763385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d4002620de4768b87ebf323f7cbb43)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:06.765603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:06.775130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5d8a9708-e400-4cd0-8c10-141fbdaf7fda None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:06.776361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5d8a9708-e400-4cd0-8c10-141fbdaf7fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:06.788770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1146/2294] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:06.793567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a9708-e400-4cd0-8c10-141fbdaf7fda None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:06.804178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a9708-e400-4cd0-8c10-141fbdaf7fda None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:06.812293 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0547-5085-417e-8729-670f315df4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:06.812966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0547-5085-417e-8729-670f315df4e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:06.813269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0547-5085-417e-8729-670f315df4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:06.813527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0547-5085-417e-8729-670f315df4e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:06.815634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c0547-5085-417e-8729-670f315df4e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:03:06.815955 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c0547-5085-417e-8729-670f315df4e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:06.820271 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c0547-5085-417e-8729-670f315df4e6 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:06.835096 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1147/2295] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:03:06.839591 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0547-5085-417e-8729-670f315df4e6 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:06.840016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d4002620de4768b87ebf323f7cbb43', 'request_id': 'req-2095fa53-c2c9-4339-9654-f851397eecdc'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:06.854051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0547-5085-417e-8729-670f315df4e6 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:06.855075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d4002620de4768b87ebf323f7cbb43', 'request_id': 'req-2095fa53-c2c9-4339-9654-f851397eecdc'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:06.859817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:06.864697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d4002620de4768b87ebf323f7cbb43', 'request_id': 'req-2095fa53-c2c9-4339-9654-f851397eecdc'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:06.884344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] The token cache is being invalidate because project 97d4002620de4768b87ebf323f7cbb43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:03:06.884645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:06.885843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:06.888955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1149/2296] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:03:06 2022] DELETE /identity/v3/projects/97d4002620de4768b87ebf323f7cbb43 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:06.931936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44a6d5-2509-45b2-b5e3-95f673d963c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:06.932859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44a6d5-2509-45b2-b5e3-95f673d963c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:06.933449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44a6d5-2509-45b2-b5e3-95f673d963c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:06.933888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44a6d5-2509-45b2-b5e3-95f673d963c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:06.934936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44a6d5-2509-45b2-b5e3-95f673d963c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:06.937421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44a6d5-2509-45b2-b5e3-95f673d963c6 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:06.973815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1148/2297] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:03:06 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:03:06.978474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:06.982790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:07.007324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095fa53-c2c9-4339-9654-f851397eecdc admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:07.016155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:07.103900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ff36b-c0aa-4028-b8b4-c69d408aec67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:07.104853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff36b-c0aa-4028-b8b4-c69d408aec67 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:07.105200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff36b-c0aa-4028-b8b4-c69d408aec67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:07.105542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff36b-c0aa-4028-b8b4-c69d408aec67 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:07.110003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ff36b-c0aa-4028-b8b4-c69d408aec67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:07.110003 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ff36b-c0aa-4028-b8b4-c69d408aec67 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:07.137899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1150/2298] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:03:06 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:03:07.142247 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44a6d5-2509-45b2-b5e3-95f673d963c6 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:07.153517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44a6d5-2509-45b2-b5e3-95f673d963c6 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:07.161401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effa3cb-4e7b-47ae-9519-2db477e6bb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:07.162096 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effa3cb-4e7b-47ae-9519-2db477e6bb8b admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:07.162427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effa3cb-4e7b-47ae-9519-2db477e6bb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:07.162717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effa3cb-4e7b-47ae-9519-2db477e6bb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:07.163462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effa3cb-4e7b-47ae-9519-2db477e6bb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:07.165088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effa3cb-4e7b-47ae-9519-2db477e6bb8b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:07.179999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1149/2299] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:03:07.183369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ff36b-c0aa-4028-b8b4-c69d408aec67 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:07.195815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ff36b-c0aa-4028-b8b4-c69d408aec67 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:07.201998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a51529-1248-4148-947b-e02b4c168b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:07.202774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51529-1248-4148-947b-e02b4c168b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:07.203064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51529-1248-4148-947b-e02b4c168b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:07.203349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51529-1248-4148-947b-e02b4c168b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:07.204087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a51529-1248-4148-947b-e02b4c168b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:07.205723 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a51529-1248-4148-947b-e02b4c168b8e admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:07.218785 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1151/2300] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:03:07.223520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effa3cb-4e7b-47ae-9519-2db477e6bb8b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:07.234354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effa3cb-4e7b-47ae-9519-2db477e6bb8b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:07.239460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531183f-3ae6-4e04-ac5d-5ca921d3da42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:07.240361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531183f-3ae6-4e04-ac5d-5ca921d3da42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:07.240870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531183f-3ae6-4e04-ac5d-5ca921d3da42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:07.241158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531183f-3ae6-4e04-ac5d-5ca921d3da42 admin admin] PATH_INFO: `/v3/projects/b20159be9d404dd2812ffeffe055ec67/users/d87d9358c42a43fea94927869c44dc82/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:07.257427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531183f-3ae6-4e04-ac5d-5ca921d3da42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b20159be9d404dd2812ffeffe055ec67, user_id=d87d9358c42a43fea94927869c44dc82, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:07.260148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531183f-3ae6-4e04-ac5d-5ca921d3da42 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:07.277555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1150/2301] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:03:07 2022] PUT /identity/v3/projects/b20159be9d404dd2812ffeffe055ec67/users/d87d9358c42a43fea94927869c44dc82/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:07.282676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64aca98-784e-4371-a990-f81637cb8dd9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:07.282971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64aca98-784e-4371-a990-f81637cb8dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:07.283265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64aca98-784e-4371-a990-f81637cb8dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:07.307791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a64aca98-784e-4371-a990-f81637cb8dd9 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:07.309033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a64aca98-784e-4371-a990-f81637cb8dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:07.377155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1152/2302] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:03:07.389401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531183f-3ae6-4e04-ac5d-5ca921d3da42 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:07.391532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9531183f-3ae6-4e04-ac5d-5ca921d3da42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:07.494469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531183f-3ae6-4e04-ac5d-5ca921d3da42 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:07.498532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffd60b-8f41-459e-9bff-3c38e3d61347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:07.499494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd60b-8f41-459e-9bff-3c38e3d61347 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:07.499827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd60b-8f41-459e-9bff-3c38e3d61347 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:07.500080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd60b-8f41-459e-9bff-3c38e3d61347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:07.514958 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffd60b-8f41-459e-9bff-3c38e3d61347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:07.518457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffd60b-8f41-459e-9bff-3c38e3d61347 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:07.537604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1151/2303] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:03:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:03:13.352407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39227e23-d4e5-4902-8fea-24dec8d542ba None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:13.352945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39227e23-d4e5-4902-8fea-24dec8d542ba None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:13.352945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39227e23-d4e5-4902-8fea-24dec8d542ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:13.397465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-39227e23-d4e5-4902-8fea-24dec8d542ba None None] MFA Rules not processed for user `d87d9358c42a43fea94927869c44dc82`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:13.398983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-39227e23-d4e5-4902-8fea-24dec8d542ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:13.509769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffd60b-8f41-459e-9bff-3c38e3d61347 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:13.515125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1153/2304] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:13.527564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffd60b-8f41-459e-9bff-3c38e3d61347 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:13.545086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce945851-951f-495b-9444-3c875aacfae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:13.545949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce945851-951f-495b-9444-3c875aacfae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:13.546425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce945851-951f-495b-9444-3c875aacfae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:13.546800 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce945851-951f-495b-9444-3c875aacfae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:13.547804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce945851-951f-495b-9444-3c875aacfae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:13.556800 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce945851-951f-495b-9444-3c875aacfae1 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:13.650971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39227e23-d4e5-4902-8fea-24dec8d542ba None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:13.658416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1152/2305] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:03:13 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:03:13.665595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39227e23-d4e5-4902-8fea-24dec8d542ba None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:13.698225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:13.827711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e6450-8d75-4a97-95b2-25e61974eba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:13.828393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6450-8d75-4a97-95b2-25e61974eba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:13.828677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6450-8d75-4a97-95b2-25e61974eba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:13.829413 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6450-8d75-4a97-95b2-25e61974eba2 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:13.829641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e6450-8d75-4a97-95b2-25e61974eba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:13.832674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e6450-8d75-4a97-95b2-25e61974eba2 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:13.856861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1154/2306] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:03:13 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:03:13.862220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce945851-951f-495b-9444-3c875aacfae1 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:13.880675 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce945851-951f-495b-9444-3c875aacfae1 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:13.890660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda5523-133c-49aa-b58f-a603de81aeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:13.891704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda5523-133c-49aa-b58f-a603de81aeff admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:13.892247 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda5523-133c-49aa-b58f-a603de81aeff admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:13.892740 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda5523-133c-49aa-b58f-a603de81aeff admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:13.893764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda5523-133c-49aa-b58f-a603de81aeff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:13.897642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda5523-133c-49aa-b58f-a603de81aeff admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:13.914624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1153/2307] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:03:13.916148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e6450-8d75-4a97-95b2-25e61974eba2 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:13.930712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e6450-8d75-4a97-95b2-25e61974eba2 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:13.937595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3be54-5efe-4e14-9563-f1ba992bd978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:13.938600 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3be54-5efe-4e14-9563-f1ba992bd978 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:13.938790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3be54-5efe-4e14-9563-f1ba992bd978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:13.939207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3be54-5efe-4e14-9563-f1ba992bd978 admin admin] PATH_INFO: `/v3/projects/f3fb15f71b4d4ccdb5e72d0804488b7e/users/cd55e696b29341f98e31cf36a34f3378/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:13.960986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3be54-5efe-4e14-9563-f1ba992bd978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3fb15f71b4d4ccdb5e72d0804488b7e, user_id=cd55e696b29341f98e31cf36a34f3378, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:13.964447 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3be54-5efe-4e14-9563-f1ba992bd978 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:13.981666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1155/2308] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:03:13 2022] PUT /identity/v3/projects/f3fb15f71b4d4ccdb5e72d0804488b7e/users/cd55e696b29341f98e31cf36a34f3378/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:13.985927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda5523-133c-49aa-b58f-a603de81aeff admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:13.999901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda5523-133c-49aa-b58f-a603de81aeff admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:14.035487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:14.074774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bef2fa-7313-4a15-9293-a3cdf2952967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:14.075835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bef2fa-7313-4a15-9293-a3cdf2952967 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:14.076311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bef2fa-7313-4a15-9293-a3cdf2952967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:14.076574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bef2fa-7313-4a15-9293-a3cdf2952967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:14.077358 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bef2fa-7313-4a15-9293-a3cdf2952967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:14.079136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bef2fa-7313-4a15-9293-a3cdf2952967 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:14.091729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1154/2309] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:03:14.095616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3be54-5efe-4e14-9563-f1ba992bd978 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:14.107737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3be54-5efe-4e14-9563-f1ba992bd978 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:14.113076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8142a98-5e19-471c-ae32-4c6924c5ab6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:14.114015 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8142a98-5e19-471c-ae32-4c6924c5ab6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:14.114261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8142a98-5e19-471c-ae32-4c6924c5ab6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:14.114565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8142a98-5e19-471c-ae32-4c6924c5ab6d admin admin] PATH_INFO: `/v3/projects/f3fb15f71b4d4ccdb5e72d0804488b7e/users/cd55e696b29341f98e31cf36a34f3378/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:14.116628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8142a98-5e19-471c-ae32-4c6924c5ab6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3fb15f71b4d4ccdb5e72d0804488b7e, user_id=cd55e696b29341f98e31cf36a34f3378, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:14.119690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8142a98-5e19-471c-ae32-4c6924c5ab6d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:14.141131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1156/2310] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:03:14 2022] PUT /identity/v3/projects/f3fb15f71b4d4ccdb5e72d0804488b7e/users/cd55e696b29341f98e31cf36a34f3378/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:03:14.463163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bef2fa-7313-4a15-9293-a3cdf2952967 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:14.465168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b6bef2fa-7313-4a15-9293-a3cdf2952967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:14.524048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bef2fa-7313-4a15-9293-a3cdf2952967 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:14.635172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:14.636076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:14.636358 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:14.636631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] PATH_INFO: `/v3/users/e0ed31ab86c34fe2bae54f1dfe3c4fb7` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:14.637971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ed31ab86c34fe2bae54f1dfe3c4fb7)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:14.640107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:14.755031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ed31ab86c34fe2bae54f1dfe3c4fb7'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:14.815505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ed31ab86c34fe2bae54f1dfe3c4fb7'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:14.830238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ed31ab86c34fe2bae54f1dfe3c4fb7'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:14.831009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ed31ab86c34fe2bae54f1dfe3c4fb7'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:14.843182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ed31ab86c34fe2bae54f1dfe3c4fb7'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:14.859713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1155/2311] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:03:14 2022] DELETE /identity/v3/users/e0ed31ab86c34fe2bae54f1dfe3c4fb7 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:14.863999 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6d5c-364f-4f0e-966d-9184c4a5d1de None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:14.864507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6d5c-364f-4f0e-966d-9184c4a5d1de None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:14.864632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6d5c-364f-4f0e-966d-9184c4a5d1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:14.919551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-1f2b6d5c-364f-4f0e-966d-9184c4a5d1de None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:14.920851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1f2b6d5c-364f-4f0e-966d-9184c4a5d1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:15.136064 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1157/2312] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:15.154904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:15.156924 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:15.280698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:15.301307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5941fd7-55f5-4427-859f-a3f91e9a9e2f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:15.305953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9685a6-b15e-470b-9650-5056e4860365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:15.306712 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9685a6-b15e-470b-9650-5056e4860365 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:15.307109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9685a6-b15e-470b-9650-5056e4860365 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:15.307487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9685a6-b15e-470b-9650-5056e4860365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:15.319433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9685a6-b15e-470b-9650-5056e4860365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:15.321091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9685a6-b15e-470b-9650-5056e4860365 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:15.337711 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1156/2313] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:03:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:03:16.199757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b6d5c-364f-4f0e-966d-9184c4a5d1de None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:16.201994 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1f2b6d5c-364f-4f0e-966d-9184c4a5d1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:16.234539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b6d5c-364f-4f0e-966d-9184c4a5d1de None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:16.251266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:16.252417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:16.252864 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:16.253145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] PATH_INFO: `/v3/projects/4102dd367da141398b035bf51b92071d` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:16.254734 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4102dd367da141398b035bf51b92071d)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:16.261432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:16.423688 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4102dd367da141398b035bf51b92071d', 'request_id': 'req-3c142582-f117-4d4f-ac3d-c8cef37b61c0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:16.439554 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4102dd367da141398b035bf51b92071d', 'request_id': 'req-3c142582-f117-4d4f-ac3d-c8cef37b61c0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:16.448642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4102dd367da141398b035bf51b92071d', 'request_id': 'req-3c142582-f117-4d4f-ac3d-c8cef37b61c0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:16.450341 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:16.475365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] The token cache is being invalidate because project 4102dd367da141398b035bf51b92071d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:03:16.475745 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:16.477185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:16.497948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9685a6-b15e-470b-9650-5056e4860365 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:16.498549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1158/2314] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:03:16 2022] DELETE /identity/v3/projects/4102dd367da141398b035bf51b92071d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:16.515085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9685a6-b15e-470b-9650-5056e4860365 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:16.616625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:16.617548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:16.618504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:16.619440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] PATH_INFO: `/v3/users/98f4486a1b824c369be44602558cb0d0` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:16.620496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f4486a1b824c369be44602558cb0d0)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:16.622539 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:16.727684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f4486a1b824c369be44602558cb0d0'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:16.742785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f4486a1b824c369be44602558cb0d0'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:16.757919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f4486a1b824c369be44602558cb0d0'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:16.758831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f4486a1b824c369be44602558cb0d0'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:16.773326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f4486a1b824c369be44602558cb0d0'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:16.784794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1157/2315] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:03:16 2022] DELETE /identity/v3/users/98f4486a1b824c369be44602558cb0d0 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:03:17.591533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:17.594464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:17.633235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c142582-f117-4d4f-ac3d-c8cef37b61c0 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:17.799571 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:17.800507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:17.800794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:17.801071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] PATH_INFO: `/v3/projects/6876e57af831480cad702138d7ee77df` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:17.806697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6876e57af831480cad702138d7ee77df)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:17.815165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:17.953540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6876e57af831480cad702138d7ee77df', 'request_id': 'req-236a10c5-86e6-4b2d-b35b-7b739f51143f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:17.966764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6876e57af831480cad702138d7ee77df', 'request_id': 'req-236a10c5-86e6-4b2d-b35b-7b739f51143f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:17.984485 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6876e57af831480cad702138d7ee77df', 'request_id': 'req-236a10c5-86e6-4b2d-b35b-7b739f51143f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.001263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] The token cache is being invalidate because project 6876e57af831480cad702138d7ee77df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:03:18.001808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.003057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.008865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1159/2316] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:03:17 2022] DELETE /identity/v3/projects/6876e57af831480cad702138d7ee77df => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:18.010115 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:18.011859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:18.030141 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f64ced-fbde-4001-8b06-1adb59278a09 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:18.143754 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061ac96-53e1-4640-ab1f-18b9b5c4cd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:18.145062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061ac96-53e1-4640-ab1f-18b9b5c4cd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:18.145505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061ac96-53e1-4640-ab1f-18b9b5c4cd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:18.146005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061ac96-53e1-4640-ab1f-18b9b5c4cd70 admin admin] PATH_INFO: `/v3/users/5d0a8d0ac7f6493baf848583a9f58d07` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:18.148420 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061ac96-53e1-4640-ab1f-18b9b5c4cd70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0a8d0ac7f6493baf848583a9f58d07)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:18.151330 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061ac96-53e1-4640-ab1f-18b9b5c4cd70 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:18.168619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:18.275990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0061ac96-53e1-4640-ab1f-18b9b5c4cd70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0a8d0ac7f6493baf848583a9f58d07'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.327808 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0061ac96-53e1-4640-ab1f-18b9b5c4cd70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0a8d0ac7f6493baf848583a9f58d07'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.344517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0061ac96-53e1-4640-ab1f-18b9b5c4cd70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0a8d0ac7f6493baf848583a9f58d07'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.345180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0061ac96-53e1-4640-ab1f-18b9b5c4cd70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0a8d0ac7f6493baf848583a9f58d07'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.367398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0061ac96-53e1-4640-ab1f-18b9b5c4cd70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0a8d0ac7f6493baf848583a9f58d07'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.432100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1158/2317] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:03:18 2022] DELETE /identity/v3/users/5d0a8d0ac7f6493baf848583a9f58d07 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:18.548857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:18.551028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:18.574900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a10c5-86e6-4b2d-b35b-7b739f51143f admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:18.698258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:18.703127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:18.703558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:18.705517 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] PATH_INFO: `/v3/users/97691a3b2ac3418993ce3af07a0baeef` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:18.707832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97691a3b2ac3418993ce3af07a0baeef)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:18.713992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:18.787549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:18.819881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97691a3b2ac3418993ce3af07a0baeef'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.833904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97691a3b2ac3418993ce3af07a0baeef'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.835651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97691a3b2ac3418993ce3af07a0baeef'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.836771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97691a3b2ac3418993ce3af07a0baeef'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.864783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97691a3b2ac3418993ce3af07a0baeef'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:18.884992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1160/2318] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:03:18 2022] DELETE /identity/v3/users/97691a3b2ac3418993ce3af07a0baeef => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:18.889250 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecadad-36d0-4940-8d76-4bde91cd7111 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:18.889549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecadad-36d0-4940-8d76-4bde91cd7111 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:18.889823 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecadad-36d0-4940-8d76-4bde91cd7111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:18.913855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-edecadad-36d0-4940-8d76-4bde91cd7111 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:18.915828 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-edecadad-36d0-4940-8d76-4bde91cd7111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:19.104449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1159/2319] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:19.174441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:19.175948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:19.381102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecadad-36d0-4940-8d76-4bde91cd7111 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:19.382799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-edecadad-36d0-4940-8d76-4bde91cd7111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:19.414230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecadad-36d0-4940-8d76-4bde91cd7111 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:19.427695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:19.428513 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:19.428806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:19.429083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] PATH_INFO: `/v3/projects/c325d6f3e99d4bb48790d23278a5986b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:19.432423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f9f8f-a7e3-4f5a-8ec2-d254eb552a63 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:19.434445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c325d6f3e99d4bb48790d23278a5986b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:19.436425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:19.445394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b045e15-37a4-405d-b276-8f44363b3e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:19.446747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b045e15-37a4-405d-b276-8f44363b3e21 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:19.447279 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b045e15-37a4-405d-b276-8f44363b3e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:19.448021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b045e15-37a4-405d-b276-8f44363b3e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:19.475890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b045e15-37a4-405d-b276-8f44363b3e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:19.481859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b045e15-37a4-405d-b276-8f44363b3e21 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:19.519942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1161/2320] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:03:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:03:19.557526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c325d6f3e99d4bb48790d23278a5986b', 'request_id': 'req-5696a675-dfe6-42ee-bd49-e5e0937c9479'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:19.572127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c325d6f3e99d4bb48790d23278a5986b', 'request_id': 'req-5696a675-dfe6-42ee-bd49-e5e0937c9479'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:19.584437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c325d6f3e99d4bb48790d23278a5986b', 'request_id': 'req-5696a675-dfe6-42ee-bd49-e5e0937c9479'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:19.612763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] The token cache is being invalidate because project c325d6f3e99d4bb48790d23278a5986b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:03:19.613154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:19.614012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5696a675-dfe6-42ee-bd49-e5e0937c9479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:19.617213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1160/2321] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:03:19 2022] DELETE /identity/v3/projects/c325d6f3e99d4bb48790d23278a5986b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:20.949875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b045e15-37a4-405d-b276-8f44363b3e21 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:20.952657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4b045e15-37a4-405d-b276-8f44363b3e21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:20.977590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b045e15-37a4-405d-b276-8f44363b3e21 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:21.114706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:21.116273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:21.116273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:21.116273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] PATH_INFO: `/v3/projects/8b33c88ff8b1492bb228167557f4630f` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:21.117945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b33c88ff8b1492bb228167557f4630f)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:21.120800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:21.180036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:21.183722 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b33c88ff8b1492bb228167557f4630f', 'request_id': 'req-a407b070-6637-4bdd-934d-2cf88d01e4f8'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:21.200270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b33c88ff8b1492bb228167557f4630f', 'request_id': 'req-a407b070-6637-4bdd-934d-2cf88d01e4f8'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:21.209123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b33c88ff8b1492bb228167557f4630f', 'request_id': 'req-a407b070-6637-4bdd-934d-2cf88d01e4f8'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:21.223641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] The token cache is being invalidate because project 8b33c88ff8b1492bb228167557f4630f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:03:21.224202 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:21.225055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:21.228617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1162/2322] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:03:20 2022] DELETE /identity/v3/projects/8b33c88ff8b1492bb228167557f4630f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:23.337850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6b360-bfc1-4936-ac9d-a0af3d8b3f4d None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:23.338368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6b360-bfc1-4936-ac9d-a0af3d8b3f4d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:23.338459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6b360-bfc1-4936-ac9d-a0af3d8b3f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:23.359350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:23.374089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-6ce6b360-bfc1-4936-ac9d-a0af3d8b3f4d None None] MFA Rules not processed for user `cd55e696b29341f98e31cf36a34f3378`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:23.380433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6ce6b360-bfc1-4936-ac9d-a0af3d8b3f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:23.493417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1161/2323] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:03:23.507527 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:23.509135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:23.648837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407b070-6637-4bdd-934d-2cf88d01e4f8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:23.665506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb87aab-dd2f-4581-aee1-927660c80064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:23.666789 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb87aab-dd2f-4581-aee1-927660c80064 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:23.666789 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb87aab-dd2f-4581-aee1-927660c80064 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:23.666789 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb87aab-dd2f-4581-aee1-927660c80064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:23.670870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6bb87aab-dd2f-4581-aee1-927660c80064 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:23.804220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb87aab-dd2f-4581-aee1-927660c80064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:23.805906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb87aab-dd2f-4581-aee1-927660c80064 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:23.839084 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1163/2324] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:03:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:03:25.223528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77f5a6-02ef-4b6d-bbda-a9479f1a64d1 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:25.224053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77f5a6-02ef-4b6d-bbda-a9479f1a64d1 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:25.224411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77f5a6-02ef-4b6d-bbda-a9479f1a64d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:25.269695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-5c77f5a6-02ef-4b6d-bbda-a9479f1a64d1 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:25.277308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5c77f5a6-02ef-4b6d-bbda-a9479f1a64d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:25.411491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1162/2325] 10.222.0.45 () {58 vars in 967 bytes} [Tue May 24 13:03:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:25.424476 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb87aab-dd2f-4581-aee1-927660c80064 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:25.426718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6bb87aab-dd2f-4581-aee1-927660c80064 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:25.552553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb87aab-dd2f-4581-aee1-927660c80064 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:25.559077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88d085-73d2-446b-bf5b-0598eed08ddc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:25.559863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88d085-73d2-446b-bf5b-0598eed08ddc service swift] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:25.572347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88d085-73d2-446b-bf5b-0598eed08ddc service swift] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:25.572765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88d085-73d2-446b-bf5b-0598eed08ddc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:25.595533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88d085-73d2-446b-bf5b-0598eed08ddc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:25.601616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88d085-73d2-446b-bf5b-0598eed08ddc service swift] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:25.621367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1164/2326] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:03:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:03:27.970684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77f5a6-02ef-4b6d-bbda-a9479f1a64d1 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:27.973052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5c77f5a6-02ef-4b6d-bbda-a9479f1a64d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:28.055385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77f5a6-02ef-4b6d-bbda-a9479f1a64d1 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:28.060030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:28.060975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:28.061286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:28.061606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] PATH_INFO: `/v3/users/226c8c0918714060ac908316c7effc08` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:28.063159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226c8c0918714060ac908316c7effc08)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:28.068370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:28.117452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:28.121962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226c8c0918714060ac908316c7effc08'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:28.137897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226c8c0918714060ac908316c7effc08'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:28.149539 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226c8c0918714060ac908316c7effc08'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:28.150261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226c8c0918714060ac908316c7effc08'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:28.157901 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226c8c0918714060ac908316c7effc08'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:28.167130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1163/2327] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:03:27 2022] DELETE /identity/v3/users/226c8c0918714060ac908316c7effc08 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:28.192050 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bf7ce-532d-46d3-96cb-0495d17fe0d1 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:28.192624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bf7ce-532d-46d3-96cb-0495d17fe0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:28.192725 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bf7ce-532d-46d3-96cb-0495d17fe0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:28.198490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:28.213264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7c1bf7ce-532d-46d3-96cb-0495d17fe0d1 None None] MFA Rules not processed for user `91a76f7af8a5467bbf567ca5b9db9ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:28.214697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7c1bf7ce-532d-46d3-96cb-0495d17fe0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:28.282492 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1165/2328] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:28.360260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:28.363006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:28.458824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1eefaf-017d-4d25-8f69-d1f65c7a17d2 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:28.463315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:28.464181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:28.464470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:28.464745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] PATH_INFO: `/v3/projects/6f2aa286c68c473686a4c71190684e18` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:28.466026 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2aa286c68c473686a4c71190684e18)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:28.467987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:28.526266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2aa286c68c473686a4c71190684e18', 'request_id': 'req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:28.535120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2aa286c68c473686a4c71190684e18', 'request_id': 'req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:28.541088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2aa286c68c473686a4c71190684e18', 'request_id': 'req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:28.552868 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] The token cache is being invalidate because project 6f2aa286c68c473686a4c71190684e18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:03:28.553154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:28.553807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af9e1da0-3e9f-43e4-bd27-69b6d00e5147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:03:28.555606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1164/2329] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:03:28 2022] DELETE /identity/v3/projects/6f2aa286c68c473686a4c71190684e18 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:03:28.560882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1235540-e256-4e4f-bf8d-f6af42ead9da None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:28.561172 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1235540-e256-4e4f-bf8d-f6af42ead9da None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:28.561928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1235540-e256-4e4f-bf8d-f6af42ead9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:28.585578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c1235540-e256-4e4f-bf8d-f6af42ead9da None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:28.586906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c1235540-e256-4e4f-bf8d-f6af42ead9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:28.651630 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1166/2330] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:03:28.666205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74ad09-fc98-4760-b99e-a5b8a6a74205 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:28.666525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74ad09-fc98-4760-b99e-a5b8a6a74205 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:28.666831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74ad09-fc98-4760-b99e-a5b8a6a74205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:28.692266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-cb74ad09-fc98-4760-b99e-a5b8a6a74205 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:28.693487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-cb74ad09-fc98-4760-b99e-a5b8a6a74205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:28.701482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1165/2331] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:28.705909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1235540-e256-4e4f-bf8d-f6af42ead9da None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:28.716136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1235540-e256-4e4f-bf8d-f6af42ead9da None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:28.722761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928c1f0-bfdf-4d68-9593-a105ba9a9135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:28.723436 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928c1f0-bfdf-4d68-9593-a105ba9a9135 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:28.723718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928c1f0-bfdf-4d68-9593-a105ba9a9135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:28.723992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928c1f0-bfdf-4d68-9593-a105ba9a9135 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:28.724763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928c1f0-bfdf-4d68-9593-a105ba9a9135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:03:28.725081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928c1f0-bfdf-4d68-9593-a105ba9a9135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:28.726720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928c1f0-bfdf-4d68-9593-a105ba9a9135 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:28.739193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1167/2332] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:03:28.743818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74ad09-fc98-4760-b99e-a5b8a6a74205 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:28.754148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74ad09-fc98-4760-b99e-a5b8a6a74205 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:28.758946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119b05e-6dfe-4e14-9c85-98890483fd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:28.759614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119b05e-6dfe-4e14-9c85-98890483fd36 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:28.759899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119b05e-6dfe-4e14-9c85-98890483fd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:28.760174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119b05e-6dfe-4e14-9c85-98890483fd36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:28.760874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119b05e-6dfe-4e14-9c85-98890483fd36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:28.762820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119b05e-6dfe-4e14-9c85-98890483fd36 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:28.795282 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1166/2333] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:03:28 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:03:28.796732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928c1f0-bfdf-4d68-9593-a105ba9a9135 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:28.807537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928c1f0-bfdf-4d68-9593-a105ba9a9135 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:28.876808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74424beb-ebe8-4de8-a592-3482983dbd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:28.877491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74424beb-ebe8-4de8-a592-3482983dbd52 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:28.877790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74424beb-ebe8-4de8-a592-3482983dbd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:28.881959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74424beb-ebe8-4de8-a592-3482983dbd52 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:28.882761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74424beb-ebe8-4de8-a592-3482983dbd52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:28.884721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74424beb-ebe8-4de8-a592-3482983dbd52 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:28.910530 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1168/2334] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:03:28 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:03:28.912159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119b05e-6dfe-4e14-9c85-98890483fd36 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:28.923155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119b05e-6dfe-4e14-9c85-98890483fd36 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:28.929481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9942e-ecb2-40cf-aea2-38a1d6ea5f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:28.930276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9942e-ecb2-40cf-aea2-38a1d6ea5f1b admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:28.930701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9942e-ecb2-40cf-aea2-38a1d6ea5f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:28.931056 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9942e-ecb2-40cf-aea2-38a1d6ea5f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:28.931856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9942e-ecb2-40cf-aea2-38a1d6ea5f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:28.933893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9942e-ecb2-40cf-aea2-38a1d6ea5f1b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:28.947371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1167/2335] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:03:28.949069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74424beb-ebe8-4de8-a592-3482983dbd52 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:28.961842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74424beb-ebe8-4de8-a592-3482983dbd52 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:28.967460 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6efa1-432e-40a8-afc9-9c51ead48b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:28.968072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6efa1-432e-40a8-afc9-9c51ead48b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:28.968354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6efa1-432e-40a8-afc9-9c51ead48b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:28.968636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6efa1-432e-40a8-afc9-9c51ead48b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:28.969361 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6efa1-432e-40a8-afc9-9c51ead48b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:28.971021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6efa1-432e-40a8-afc9-9c51ead48b08 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:28.985628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1169/2336] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:03:28.986640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9942e-ecb2-40cf-aea2-38a1d6ea5f1b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:28.998048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9942e-ecb2-40cf-aea2-38a1d6ea5f1b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:29.002793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13d9cd-1501-4984-9b9c-f2d436327e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:29.003786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13d9cd-1501-4984-9b9c-f2d436327e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:29.004150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13d9cd-1501-4984-9b9c-f2d436327e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:29.004503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13d9cd-1501-4984-9b9c-f2d436327e52 admin admin] PATH_INFO: `/v3/projects/c6c32c343c20445893570de4b5c9145d/users/cbfa40eee81742f7bc51145a56cad25d/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:29.008701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:29.025195 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13d9cd-1501-4984-9b9c-f2d436327e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c32c343c20445893570de4b5c9145d, user_id=cbfa40eee81742f7bc51145a56cad25d, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:29.029357 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13d9cd-1501-4984-9b9c-f2d436327e52 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:29.045980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1168/2337] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:03:28 2022] PUT /identity/v3/projects/c6c32c343c20445893570de4b5c9145d/users/cbfa40eee81742f7bc51145a56cad25d/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:03:29.056843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a6ab9-6296-4624-9d2b-420f49f66f06 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:29.057134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a6ab9-6296-4624-9d2b-420f49f66f06 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:29.057426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a6ab9-6296-4624-9d2b-420f49f66f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:29.087740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:29.091686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-722a6ab9-6296-4624-9d2b-420f49f66f06 None None] MFA Rules not processed for user `cbfa40eee81742f7bc51145a56cad25d`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:29.093957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-722a6ab9-6296-4624-9d2b-420f49f66f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:29.164963 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1170/2338] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:03:29.179972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13d9cd-1501-4984-9b9c-f2d436327e52 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:03:29.180267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3a13d9cd-1501-4984-9b9c-f2d436327e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:29.275725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13d9cd-1501-4984-9b9c-f2d436327e52 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:03:29.280860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828af9-1f55-4f3e-8650-0a4705527469 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e47f1d65687e4cfcbce764076141702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:03:29.281529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828af9-1f55-4f3e-8650-0a4705527469 service glance] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:29.281821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828af9-1f55-4f3e-8650-0a4705527469 service glance] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:29.282231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828af9-1f55-4f3e-8650-0a4705527469 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:29.292454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a828af9-1f55-4f3e-8650-0a4705527469 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:03:29.294095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a828af9-1f55-4f3e-8650-0a4705527469 service glance] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:03:29.303574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1169/2339] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:03:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:03:39.161575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=26 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:39.299302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:03:55.158058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc929a3-990f-4681-b107-c7128a820f6a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:03:55.158646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc929a3-990f-4681-b107-c7128a820f6a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:03:55.159310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc929a3-990f-4681-b107-c7128a820f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:03:55.176366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-bcc929a3-990f-4681-b107-c7128a820f6a None None] MFA Rules not processed for user `d87d9358c42a43fea94927869c44dc82`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:03:55.177529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bcc929a3-990f-4681-b107-c7128a820f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:03:55.229076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1171/2340] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:03:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:04:05.222687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=25 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:04:10.661442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828af9-1f55-4f3e-8650-0a4705527469 service glance] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:10.723709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828af9-1f55-4f3e-8650-0a4705527469 service glance] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:10.728148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:10.729008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:10.729287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:10.729559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] PATH_INFO: `/v3/users/cbfa40eee81742f7bc51145a56cad25d` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:10.730775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbfa40eee81742f7bc51145a56cad25d)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:10.732695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:10.776870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbfa40eee81742f7bc51145a56cad25d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:10.789350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbfa40eee81742f7bc51145a56cad25d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:10.796291 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbfa40eee81742f7bc51145a56cad25d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:10.797023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbfa40eee81742f7bc51145a56cad25d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:10.803829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbfa40eee81742f7bc51145a56cad25d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:10.809651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1170/2341] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:04:10 2022] DELETE /identity/v3/users/cbfa40eee81742f7bc51145a56cad25d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:04:10.814351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d91cf-7eb5-4d7d-b0c3-262788084b4c None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:10.814832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d91cf-7eb5-4d7d-b0c3-262788084b4c None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:10.814943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d91cf-7eb5-4d7d-b0c3-262788084b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:10.832288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ac3d91cf-7eb5-4d7d-b0c3-262788084b4c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:04:10.833457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ac3d91cf-7eb5-4d7d-b0c3-262788084b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:10.888457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1172/2342] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:04:10.898410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:10.899804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:10.956084 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77199745-474e-4dd1-bfaf-da956a85f11b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:10.959629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721cd51-3a7b-4e41-a1f2-bd08ebfd36a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:10.960266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721cd51-3a7b-4e41-a1f2-bd08ebfd36a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:10.960549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721cd51-3a7b-4e41-a1f2-bd08ebfd36a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:10.960824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721cd51-3a7b-4e41-a1f2-bd08ebfd36a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:10.969501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721cd51-3a7b-4e41-a1f2-bd08ebfd36a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:10.970984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721cd51-3a7b-4e41-a1f2-bd08ebfd36a8 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:10.980283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1171/2343] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:04:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:04:11.258908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d91cf-7eb5-4d7d-b0c3-262788084b4c None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:11.260436 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ac3d91cf-7eb5-4d7d-b0c3-262788084b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:11.273223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d91cf-7eb5-4d7d-b0c3-262788084b4c None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:11.279178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:11.280042 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:11.280350 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:11.280622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] PATH_INFO: `/v3/projects/c6c32c343c20445893570de4b5c9145d` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:11.281867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6c32c343c20445893570de4b5c9145d)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:11.284236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:11.340040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6c32c343c20445893570de4b5c9145d', 'request_id': 'req-4c345914-921f-433f-94e4-6b0725256e4f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:11.347833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6c32c343c20445893570de4b5c9145d', 'request_id': 'req-4c345914-921f-433f-94e4-6b0725256e4f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:11.353525 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6c32c343c20445893570de4b5c9145d', 'request_id': 'req-4c345914-921f-433f-94e4-6b0725256e4f'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:11.364976 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] The token cache is being invalidate because project c6c32c343c20445893570de4b5c9145d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:04:11.365269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:11.365895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4c345914-921f-433f-94e4-6b0725256e4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:11.367718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1173/2344] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:04:11 2022] DELETE /identity/v3/projects/c6c32c343c20445893570de4b5c9145d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:04:11.372601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42125d28-e978-406d-9e8c-62ee8f763626 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:11.372896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42125d28-e978-406d-9e8c-62ee8f763626 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:11.373181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42125d28-e978-406d-9e8c-62ee8f763626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:11.390537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-42125d28-e978-406d-9e8c-62ee8f763626 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:04:11.391733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-42125d28-e978-406d-9e8c-62ee8f763626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:11.410006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:04:11.453365 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1172/2345] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:04:11.461553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d73d7-13f1-4dde-b0c5-7018ec74d64f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:11.461812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d73d7-13f1-4dde-b0c5-7018ec74d64f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:11.462097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d73d7-13f1-4dde-b0c5-7018ec74d64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:11.480932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-1a2d73d7-13f1-4dde-b0c5-7018ec74d64f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:04:11.482138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1a2d73d7-13f1-4dde-b0c5-7018ec74d64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:11.489945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1174/2346] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:04:11.494478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42125d28-e978-406d-9e8c-62ee8f763626 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:11.504148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42125d28-e978-406d-9e8c-62ee8f763626 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:11.508670 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b864f94-5577-4aa0-b9d9-40054e19ee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:11.509325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b864f94-5577-4aa0-b9d9-40054e19ee3b admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:11.509613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b864f94-5577-4aa0-b9d9-40054e19ee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:11.509894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b864f94-5577-4aa0-b9d9-40054e19ee3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:11.510702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b864f94-5577-4aa0-b9d9-40054e19ee3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:04:11.511013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b864f94-5577-4aa0-b9d9-40054e19ee3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:11.512620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b864f94-5577-4aa0-b9d9-40054e19ee3b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:11.524645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1173/2347] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:04:11.528860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d73d7-13f1-4dde-b0c5-7018ec74d64f None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:11.538741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d73d7-13f1-4dde-b0c5-7018ec74d64f None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:11.543512 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36529d-c5f1-466c-8c77-2da7eff201d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:11.544189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36529d-c5f1-466c-8c77-2da7eff201d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:11.544473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36529d-c5f1-466c-8c77-2da7eff201d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:11.544749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36529d-c5f1-466c-8c77-2da7eff201d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:11.545435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36529d-c5f1-466c-8c77-2da7eff201d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:11.547416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36529d-c5f1-466c-8c77-2da7eff201d8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:11.572499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1175/2348] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:04:11 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:04:11.576574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b864f94-5577-4aa0-b9d9-40054e19ee3b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:11.587193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b864f94-5577-4aa0-b9d9-40054e19ee3b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:11.647687 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8e312-c0f7-4f7d-a3de-efa7700cb375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:11.648358 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8e312-c0f7-4f7d-a3de-efa7700cb375 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:11.648642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8e312-c0f7-4f7d-a3de-efa7700cb375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:11.648918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8e312-c0f7-4f7d-a3de-efa7700cb375 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:11.649643 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8e312-c0f7-4f7d-a3de-efa7700cb375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:11.651583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8e312-c0f7-4f7d-a3de-efa7700cb375 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:11.675913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1174/2349] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:04:11 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:04:11.677536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36529d-c5f1-466c-8c77-2da7eff201d8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:11.687860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36529d-c5f1-466c-8c77-2da7eff201d8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:11.692554 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba982ede-9679-4007-8cb5-dc28dd238e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:11.693223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba982ede-9679-4007-8cb5-dc28dd238e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:11.693511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba982ede-9679-4007-8cb5-dc28dd238e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:11.693792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba982ede-9679-4007-8cb5-dc28dd238e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:11.694555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba982ede-9679-4007-8cb5-dc28dd238e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:11.696163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba982ede-9679-4007-8cb5-dc28dd238e6a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:11.708848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1176/2350] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:04:11.713167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8e312-c0f7-4f7d-a3de-efa7700cb375 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:11.723601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8e312-c0f7-4f7d-a3de-efa7700cb375 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:11.728171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea0e8e-4fa1-4f54-b2f9-5f39d5eb5072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:11.728826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea0e8e-4fa1-4f54-b2f9-5f39d5eb5072 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:11.729108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea0e8e-4fa1-4f54-b2f9-5f39d5eb5072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:11.729383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea0e8e-4fa1-4f54-b2f9-5f39d5eb5072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:11.730109 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea0e8e-4fa1-4f54-b2f9-5f39d5eb5072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:11.731732 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea0e8e-4fa1-4f54-b2f9-5f39d5eb5072 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:11.741946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1175/2351] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:04:11.746622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba982ede-9679-4007-8cb5-dc28dd238e6a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:11.756768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba982ede-9679-4007-8cb5-dc28dd238e6a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:11.762580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b8705-a58c-4db3-9804-60ad493ab569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:11.763458 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b8705-a58c-4db3-9804-60ad493ab569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:11.763736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b8705-a58c-4db3-9804-60ad493ab569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:11.764008 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b8705-a58c-4db3-9804-60ad493ab569 admin admin] PATH_INFO: `/v3/projects/f2c95a8e0e284a38ab7a4f699b80c285/users/9ddec7b35f264b1caf17cc8db57ff702/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:11.779718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b8705-a58c-4db3-9804-60ad493ab569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c95a8e0e284a38ab7a4f699b80c285, user_id=9ddec7b35f264b1caf17cc8db57ff702, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:11.784110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b8705-a58c-4db3-9804-60ad493ab569 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:11.797545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1177/2352] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:04:11 2022] PUT /identity/v3/projects/f2c95a8e0e284a38ab7a4f699b80c285/users/9ddec7b35f264b1caf17cc8db57ff702/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:04:11.809738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daeeb8-453a-4a6f-b49c-5b27edf48ec2 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:11.810029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daeeb8-453a-4a6f-b49c-5b27edf48ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:11.810338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daeeb8-453a-4a6f-b49c-5b27edf48ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:11.839494 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a0daeeb8-453a-4a6f-b49c-5b27edf48ec2 None None] MFA Rules not processed for user `9ddec7b35f264b1caf17cc8db57ff702`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:04:11.840722 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a0daeeb8-453a-4a6f-b49c-5b27edf48ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:11.903718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1176/2353] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:04:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:04:11.915438 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b8705-a58c-4db3-9804-60ad493ab569 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:11.917279 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-384b8705-a58c-4db3-9804-60ad493ab569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:11.939027 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:04:11.976511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b8705-a58c-4db3-9804-60ad493ab569 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:11.980101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f76a8e-638a-4233-b286-21e7e28e5873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:11.980746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f76a8e-638a-4233-b286-21e7e28e5873 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:11.981029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f76a8e-638a-4233-b286-21e7e28e5873 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:11.981303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f76a8e-638a-4233-b286-21e7e28e5873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:11.989766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f76a8e-638a-4233-b286-21e7e28e5873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:11.991235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f76a8e-638a-4233-b286-21e7e28e5873 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:12.000754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1178/2354] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:04:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:04:12.586930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daeeb8-453a-4a6f-b49c-5b27edf48ec2 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:12.588306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a0daeeb8-453a-4a6f-b49c-5b27edf48ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:12.608195 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:04:12.645721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daeeb8-453a-4a6f-b49c-5b27edf48ec2 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:12.649231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b74fe-5a80-4977-9ec7-39b53bdbb539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:12.649877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b74fe-5a80-4977-9ec7-39b53bdbb539 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:12.650161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b74fe-5a80-4977-9ec7-39b53bdbb539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:12.650471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b74fe-5a80-4977-9ec7-39b53bdbb539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:12.652270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-975b74fe-5a80-4977-9ec7-39b53bdbb539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:12.721567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b74fe-5a80-4977-9ec7-39b53bdbb539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:12.723049 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b74fe-5a80-4977-9ec7-39b53bdbb539 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:12.732590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1177/2355] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:04:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:04:21.792917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f76a8e-638a-4233-b286-21e7e28e5873 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:21.857164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f76a8e-638a-4233-b286-21e7e28e5873 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:21.862276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:21.863181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:21.863466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:21.863743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] PATH_INFO: `/v3/users/9ddec7b35f264b1caf17cc8db57ff702` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:21.864958 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ddec7b35f264b1caf17cc8db57ff702)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:21.866957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:21.912380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ddec7b35f264b1caf17cc8db57ff702'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:21.919262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ddec7b35f264b1caf17cc8db57ff702'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:21.920481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ddec7b35f264b1caf17cc8db57ff702'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:21.921136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ddec7b35f264b1caf17cc8db57ff702'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:21.933579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ddec7b35f264b1caf17cc8db57ff702'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:21.945420 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1179/2356] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:04:21 2022] DELETE /identity/v3/users/9ddec7b35f264b1caf17cc8db57ff702 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:04:21.950136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6edd7-00aa-49f9-be7f-2a8304a8bd63 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:21.950655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6edd7-00aa-49f9-be7f-2a8304a8bd63 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:21.950756 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6edd7-00aa-49f9-be7f-2a8304a8bd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:21.967961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-61b6edd7-00aa-49f9-be7f-2a8304a8bd63 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:04:21.969163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-61b6edd7-00aa-49f9-be7f-2a8304a8bd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:22.026797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1178/2357] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:04:22.037051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:22.038508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:22.084968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:04:22.098168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d918007-cce2-41a6-8472-39c1c0f9f82a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:22.101796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674d0d2-9c62-4d67-b1c4-9e3d8caf0e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:22.102469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674d0d2-9c62-4d67-b1c4-9e3d8caf0e6d service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:22.102754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674d0d2-9c62-4d67-b1c4-9e3d8caf0e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:22.103142 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674d0d2-9c62-4d67-b1c4-9e3d8caf0e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:22.112067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674d0d2-9c62-4d67-b1c4-9e3d8caf0e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:22.113539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674d0d2-9c62-4d67-b1c4-9e3d8caf0e6d service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:22.123147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1180/2358] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:04:22.391857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6edd7-00aa-49f9-be7f-2a8304a8bd63 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:22.393295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-61b6edd7-00aa-49f9-be7f-2a8304a8bd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:22.409842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6edd7-00aa-49f9-be7f-2a8304a8bd63 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:22.414620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:22.415486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:22.415769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:22.416066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] PATH_INFO: `/v3/projects/f2c95a8e0e284a38ab7a4f699b80c285` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:22.417221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c95a8e0e284a38ab7a4f699b80c285)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:22.419167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:22.465713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c95a8e0e284a38ab7a4f699b80c285', 'request_id': 'req-16951022-76e4-470a-9147-38df5264d339'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:22.473361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c95a8e0e284a38ab7a4f699b80c285', 'request_id': 'req-16951022-76e4-470a-9147-38df5264d339'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:22.478310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:04:22.479498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c95a8e0e284a38ab7a4f699b80c285', 'request_id': 'req-16951022-76e4-470a-9147-38df5264d339'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:22.491977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] The token cache is being invalidate because project f2c95a8e0e284a38ab7a4f699b80c285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:04:22.492521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:22.493153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-16951022-76e4-470a-9147-38df5264d339 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:22.494704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1179/2359] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:04:22 2022] DELETE /identity/v3/projects/f2c95a8e0e284a38ab7a4f699b80c285 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:04:22.499630 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1583549-efb2-4791-ba09-7f123c89aa96 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:22.499935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1583549-efb2-4791-ba09-7f123c89aa96 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:22.500198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1583549-efb2-4791-ba09-7f123c89aa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:22.518246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a1583549-efb2-4791-ba09-7f123c89aa96 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:04:22.519491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a1583549-efb2-4791-ba09-7f123c89aa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:22.586845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1181/2360] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:04:22.594401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b531-2b63-4327-a9ad-367a8bd8f4f6 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:22.594696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b531-2b63-4327-a9ad-367a8bd8f4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:22.595036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b531-2b63-4327-a9ad-367a8bd8f4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:22.613032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a872b531-2b63-4327-a9ad-367a8bd8f4f6 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:04:22.614232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a872b531-2b63-4327-a9ad-367a8bd8f4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:22.622590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1180/2361] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:04:22.627144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1583549-efb2-4791-ba09-7f123c89aa96 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:22.637490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1583549-efb2-4791-ba09-7f123c89aa96 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:22.643349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee8deb-77b6-449e-928c-eab6f62485b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:22.644105 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee8deb-77b6-449e-928c-eab6f62485b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:22.644386 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee8deb-77b6-449e-928c-eab6f62485b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:22.644666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee8deb-77b6-449e-928c-eab6f62485b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:22.645388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee8deb-77b6-449e-928c-eab6f62485b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:04:22.645700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee8deb-77b6-449e-928c-eab6f62485b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:22.647446 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee8deb-77b6-449e-928c-eab6f62485b3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:22.661854 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1182/2362] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:04:22.666742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872b531-2b63-4327-a9ad-367a8bd8f4f6 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:22.676861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872b531-2b63-4327-a9ad-367a8bd8f4f6 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:22.681538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97e07b-c705-4ade-84fa-c4efb99daa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:22.682197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97e07b-c705-4ade-84fa-c4efb99daa6f admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:22.682511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97e07b-c705-4ade-84fa-c4efb99daa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:22.682789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97e07b-c705-4ade-84fa-c4efb99daa6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:22.683552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97e07b-c705-4ade-84fa-c4efb99daa6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:22.685515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97e07b-c705-4ade-84fa-c4efb99daa6f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:22.717351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1181/2363] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:04:22 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:04:22.722138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee8deb-77b6-449e-928c-eab6f62485b3 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:22.733749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee8deb-77b6-449e-928c-eab6f62485b3 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:22.800133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfad86-47b8-4ac3-ab1e-4f89c3648087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:22.800802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfad86-47b8-4ac3-ab1e-4f89c3648087 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:22.801085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfad86-47b8-4ac3-ab1e-4f89c3648087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:22.801423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfad86-47b8-4ac3-ab1e-4f89c3648087 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:22.802039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dfad86-47b8-4ac3-ab1e-4f89c3648087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:22.804031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dfad86-47b8-4ac3-ab1e-4f89c3648087 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:22.831201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1183/2364] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:04:22 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:04:22.835324 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97e07b-c705-4ade-84fa-c4efb99daa6f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:22.845677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97e07b-c705-4ade-84fa-c4efb99daa6f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:22.850200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4c6aa-c848-43db-bfee-03aa4defd58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:22.850881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4c6aa-c848-43db-bfee-03aa4defd58b admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:22.851163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4c6aa-c848-43db-bfee-03aa4defd58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:22.851437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4c6aa-c848-43db-bfee-03aa4defd58b admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:22.852156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4c6aa-c848-43db-bfee-03aa4defd58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:22.853826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4c6aa-c848-43db-bfee-03aa4defd58b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:22.867679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1182/2365] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:04:22.871629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfad86-47b8-4ac3-ab1e-4f89c3648087 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:22.882747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfad86-47b8-4ac3-ab1e-4f89c3648087 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:22.887989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a188a-24f8-4333-b896-e8be4b5f1986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:22.888654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a188a-24f8-4333-b896-e8be4b5f1986 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:22.888941 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a188a-24f8-4333-b896-e8be4b5f1986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:22.889218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a188a-24f8-4333-b896-e8be4b5f1986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:22.889975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a188a-24f8-4333-b896-e8be4b5f1986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:22.891614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a188a-24f8-4333-b896-e8be4b5f1986 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:22.901560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1184/2366] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:04:22.905699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4c6aa-c848-43db-bfee-03aa4defd58b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:22.915656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4c6aa-c848-43db-bfee-03aa4defd58b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:22.920225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f61cec-5c3a-42c2-a413-be43bc22c18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:22.921133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f61cec-5c3a-42c2-a413-be43bc22c18f admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:22.921417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f61cec-5c3a-42c2-a413-be43bc22c18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:22.921694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f61cec-5c3a-42c2-a413-be43bc22c18f admin admin] PATH_INFO: `/v3/projects/4e98da1a717b43f9bde0fca3b2cd9b97/users/afd3b9602fbf404ca9c212b92127fe45/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:22.936962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f61cec-5c3a-42c2-a413-be43bc22c18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e98da1a717b43f9bde0fca3b2cd9b97, user_id=afd3b9602fbf404ca9c212b92127fe45, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:22.940322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f61cec-5c3a-42c2-a413-be43bc22c18f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:22.952405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1183/2367] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:04:22 2022] PUT /identity/v3/projects/4e98da1a717b43f9bde0fca3b2cd9b97/users/afd3b9602fbf404ca9c212b92127fe45/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:04:22.963418 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e16ec8-afd5-480a-b4d6-042d1d959fd0 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:22.963710 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e16ec8-afd5-480a-b4d6-042d1d959fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:22.963989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e16ec8-afd5-480a-b4d6-042d1d959fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:22.990103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-56e16ec8-afd5-480a-b4d6-042d1d959fd0 None None] MFA Rules not processed for user `afd3b9602fbf404ca9c212b92127fe45`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:04:22.991273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-56e16ec8-afd5-480a-b4d6-042d1d959fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:23.030167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:04:23.055110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1185/2368] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:04:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:04:23.067144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f61cec-5c3a-42c2-a413-be43bc22c18f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:23.068864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-38f61cec-5c3a-42c2-a413-be43bc22c18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:23.125305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f61cec-5c3a-42c2-a413-be43bc22c18f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:23.128977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e9024-8d61-462e-b15f-e18acc3737b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:23.129618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e9024-8d61-462e-b15f-e18acc3737b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:23.129903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e9024-8d61-462e-b15f-e18acc3737b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:23.130186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e9024-8d61-462e-b15f-e18acc3737b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:23.138477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e9024-8d61-462e-b15f-e18acc3737b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:23.139904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e9024-8d61-462e-b15f-e18acc3737b7 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:23.148579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1184/2369] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:04:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:04:32.209594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e16ec8-afd5-480a-b4d6-042d1d959fd0 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:32.271732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e16ec8-afd5-480a-b4d6-042d1d959fd0 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:32.276611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:32.277535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:32.277819 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:32.278094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] PATH_INFO: `/v3/users/afd3b9602fbf404ca9c212b92127fe45` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:32.279346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd3b9602fbf404ca9c212b92127fe45)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:32.281315 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:32.327683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd3b9602fbf404ca9c212b92127fe45'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:32.335716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd3b9602fbf404ca9c212b92127fe45'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:32.336283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd3b9602fbf404ca9c212b92127fe45'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:32.337104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd3b9602fbf404ca9c212b92127fe45'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:32.349727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd3b9602fbf404ca9c212b92127fe45'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:32.362081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1186/2370] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:04:32 2022] DELETE /identity/v3/users/afd3b9602fbf404ca9c212b92127fe45 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:04:32.366605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea97fe-dcea-44e8-87f6-66bf328284ea None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:32.367353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea97fe-dcea-44e8-87f6-66bf328284ea None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:32.367444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea97fe-dcea-44e8-87f6-66bf328284ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:32.384515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-b8ea97fe-dcea-44e8-87f6-66bf328284ea None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:04:32.385651 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b8ea97fe-dcea-44e8-87f6-66bf328284ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:32.453958 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1185/2371] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:04:32.463357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:32.464760 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:32.521909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac6970-1a83-4c4c-ac20-fb51dac123f0 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:32.525544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e1e20-27fd-4820-a1a0-c560411314a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:32.526186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e1e20-27fd-4820-a1a0-c560411314a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:32.526508 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e1e20-27fd-4820-a1a0-c560411314a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:32.526786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e1e20-27fd-4820-a1a0-c560411314a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:32.536682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e1e20-27fd-4820-a1a0-c560411314a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:32.538706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e1e20-27fd-4820-a1a0-c560411314a4 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:32.549228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1187/2372] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:04:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:04:32.806374 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea97fe-dcea-44e8-87f6-66bf328284ea None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:32.807857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b8ea97fe-dcea-44e8-87f6-66bf328284ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:32.820839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea97fe-dcea-44e8-87f6-66bf328284ea None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:32.825366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:32.826226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:32.826544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:32.826831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] PATH_INFO: `/v3/projects/4e98da1a717b43f9bde0fca3b2cd9b97` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:32.827996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e98da1a717b43f9bde0fca3b2cd9b97)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:32.831179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:32.860529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:04:32.879263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e98da1a717b43f9bde0fca3b2cd9b97', 'request_id': 'req-39ee5e4b-0257-4ab5-b938-a101acf2cc57'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:32.886206 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e98da1a717b43f9bde0fca3b2cd9b97', 'request_id': 'req-39ee5e4b-0257-4ab5-b938-a101acf2cc57'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:32.892556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e98da1a717b43f9bde0fca3b2cd9b97', 'request_id': 'req-39ee5e4b-0257-4ab5-b938-a101acf2cc57'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:32.904542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] The token cache is being invalidate because project 4e98da1a717b43f9bde0fca3b2cd9b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:04:32.904838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:32.905492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:04:32.907182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1186/2373] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:04:32 2022] DELETE /identity/v3/projects/4e98da1a717b43f9bde0fca3b2cd9b97 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:04:42.544420 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:04:42.901086 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:04:50.561834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f17f5d-94a0-4c37-8302-a2bc190da1ec None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:50.562346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f17f5d-94a0-4c37-8302-a2bc190da1ec None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:50.562462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f17f5d-94a0-4c37-8302-a2bc190da1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:50.580944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b9f17f5d-94a0-4c37-8302-a2bc190da1ec None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:04:50.582172 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b9f17f5d-94a0-4c37-8302-a2bc190da1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:50.636344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1188/2374] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 13:04:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:04:50.683548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:50.684996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:50.759434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee5e4b-0257-4ab5-b938-a101acf2cc57 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:50.765819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1facf33-debb-4238-bb9e-c6fa099b6b8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:50.766525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1facf33-debb-4238-bb9e-c6fa099b6b8e service swift] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:50.766819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1facf33-debb-4238-bb9e-c6fa099b6b8e service swift] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:50.767101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1facf33-debb-4238-bb9e-c6fa099b6b8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:50.776018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1facf33-debb-4238-bb9e-c6fa099b6b8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:50.777653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1facf33-debb-4238-bb9e-c6fa099b6b8e service swift] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:50.786389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1187/2375] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:04:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:04:58.375614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f17f5d-94a0-4c37-8302-a2bc190da1ec None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:58.377015 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b9f17f5d-94a0-4c37-8302-a2bc190da1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:58.452883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f17f5d-94a0-4c37-8302-a2bc190da1ec None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:58.456897 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce780fc-c2f5-411e-a205-57e1dc685d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:58.457891 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce780fc-c2f5-411e-a205-57e1dc685d19 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:58.458118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce780fc-c2f5-411e-a205-57e1dc685d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:58.459102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce780fc-c2f5-411e-a205-57e1dc685d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:58.461043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7ce780fc-c2f5-411e-a205-57e1dc685d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:58.544576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce780fc-c2f5-411e-a205-57e1dc685d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:58.546195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce780fc-c2f5-411e-a205-57e1dc685d19 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:58.574817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1189/2376] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:04:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:04:59.756957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1facf33-debb-4238-bb9e-c6fa099b6b8e service swift] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:04:59.758590 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c1facf33-debb-4238-bb9e-c6fa099b6b8e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:59.771969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1facf33-debb-4238-bb9e-c6fa099b6b8e service swift] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:04:59.775911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6afa83-4b62-4631-8d01-9233a26e62d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:04:59.776563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6afa83-4b62-4631-8d01-9233a26e62d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:04:59.776854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6afa83-4b62-4631-8d01-9233a26e62d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:04:59.777121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6afa83-4b62-4631-8d01-9233a26e62d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:04:59.778977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-db6afa83-4b62-4631-8d01-9233a26e62d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:04:59.795122 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6afa83-4b62-4631-8d01-9233a26e62d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:04:59.795343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6afa83-4b62-4631-8d01-9233a26e62d2 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:04:59.807965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1188/2377] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:04:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:05:05.160121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d341f-0467-44ae-b5e8-a6f0fa806926 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:05.160670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d341f-0467-44ae-b5e8-a6f0fa806926 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:05.160798 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d341f-0467-44ae-b5e8-a6f0fa806926 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:05.162271 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1190/2378] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 13:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:05:05.168834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e9c91-3b78-4569-b66d-4a2a4967b6aa None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:05.169316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e9c91-3b78-4569-b66d-4a2a4967b6aa None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:05.169636 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e9c91-3b78-4569-b66d-4a2a4967b6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:05.207428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-f82e9c91-3b78-4569-b66d-4a2a4967b6aa None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:05:05.208652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f82e9c91-3b78-4569-b66d-4a2a4967b6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:05.217971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1189/2379] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 13:05:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:05:05.229997 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d341f-0467-44ae-b5e8-a6f0fa806926 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:05:05.231862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6d4d341f-0467-44ae-b5e8-a6f0fa806926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:05.287189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:05:05.293811 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d341f-0467-44ae-b5e8-a6f0fa806926 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:05:05.297555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c021d-b334-4b55-a69b-1cffe681b92c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e47f1d65687e4cfcbce764076141702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:05:05.298198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c021d-b334-4b55-a69b-1cffe681b92c service glance] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:05.298509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c021d-b334-4b55-a69b-1cffe681b92c service glance] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:05.298781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c021d-b334-4b55-a69b-1cffe681b92c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:05.308590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c021d-b334-4b55-a69b-1cffe681b92c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:05:05.310042 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c021d-b334-4b55-a69b-1cffe681b92c service glance] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:05:05.320937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1191/2380] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:05:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:05:05.389074 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4dafa5-4492-4e51-9986-e0b7b31f72eb None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:05.389283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4dafa5-4492-4e51-9986-e0b7b31f72eb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:05.389586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4dafa5-4492-4e51-9986-e0b7b31f72eb None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:05.391024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1190/2381] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 13:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:05:05.395667 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbc4df-5ff2-42bf-83cf-755fc0680d21 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:05.395986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbc4df-5ff2-42bf-83cf-755fc0680d21 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:05.396266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbc4df-5ff2-42bf-83cf-755fc0680d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:05.431768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b8cbc4df-5ff2-42bf-83cf-755fc0680d21 None None] MFA Rules not processed for user `bb111be6bc2946d3b50b8d99a89cf8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:05:05.432955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b8cbc4df-5ff2-42bf-83cf-755fc0680d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:05.439451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1192/2382] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 13:05:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:05:05.451720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4dafa5-4492-4e51-9986-e0b7b31f72eb None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:05:05.462623 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4dafa5-4492-4e51-9986-e0b7b31f72eb None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:05:05.466254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a9400-81b7-46d2-b43e-3a7e1188109b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:05:05.466927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a9400-81b7-46d2-b43e-3a7e1188109b service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:05.467216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a9400-81b7-46d2-b43e-3a7e1188109b service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:05.467500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a9400-81b7-46d2-b43e-3a7e1188109b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:05.474883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a9400-81b7-46d2-b43e-3a7e1188109b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:05:05.476332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a9400-81b7-46d2-b43e-3a7e1188109b service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:05:05.484062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1191/2383] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:05:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:05:05.566944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec36600-bbb4-4bdb-b040-5c6cfbc74808 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:05.567234 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec36600-bbb4-4bdb-b040-5c6cfbc74808 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:05.567514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec36600-bbb4-4bdb-b040-5c6cfbc74808 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:05.568900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1193/2384] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 13:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:05:05.574564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762136-2672-4d16-8bbd-5e411223aaf6 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:05.574858 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762136-2672-4d16-8bbd-5e411223aaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:05.575135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762136-2672-4d16-8bbd-5e411223aaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:05.593212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-85762136-2672-4d16-8bbd-5e411223aaf6 None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:05:05.594454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-85762136-2672-4d16-8bbd-5e411223aaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:05.603675 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1192/2385] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 13:05:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:05:05.614276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec36600-bbb4-4bdb-b040-5c6cfbc74808 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:05:05.625374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec36600-bbb4-4bdb-b040-5c6cfbc74808 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:05:05.628985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f379e3-c585-4e68-8dcb-ddd20fcf7808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:05:05.629618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f379e3-c585-4e68-8dcb-ddd20fcf7808 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:05.629909 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f379e3-c585-4e68-8dcb-ddd20fcf7808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:05.630187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f379e3-c585-4e68-8dcb-ddd20fcf7808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:05.643506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f379e3-c585-4e68-8dcb-ddd20fcf7808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:05:05.644877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f379e3-c585-4e68-8dcb-ddd20fcf7808 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:05:05.655901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1194/2386] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:05:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:05:05.716445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85762136-2672-4d16-8bbd-5e411223aaf6 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:05:05.717829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-85762136-2672-4d16-8bbd-5e411223aaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:05.786351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85762136-2672-4d16-8bbd-5e411223aaf6 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:05:05.791401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43e62f-320c-460b-9236-ff4c30c75ad2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:05:05.792047 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43e62f-320c-460b-9236-ff4c30c75ad2 service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:05.792336 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43e62f-320c-460b-9236-ff4c30c75ad2 service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:05.792618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43e62f-320c-460b-9236-ff4c30c75ad2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:05.794404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3c43e62f-320c-460b-9236-ff4c30c75ad2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:05.806173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43e62f-320c-460b-9236-ff4c30c75ad2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:05:05.807826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43e62f-320c-460b-9236-ff4c30c75ad2 service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:05:05.817490 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1193/2387] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:05:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:05:08.022055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aa364-1b98-4556-8c4b-a4d0b524624d None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:08.022589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aa364-1b98-4556-8c4b-a4d0b524624d None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:08.022932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aa364-1b98-4556-8c4b-a4d0b524624d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:08.041095 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-2d4aa364-1b98-4556-8c4b-a4d0b524624d None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:05:08.042416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2d4aa364-1b98-4556-8c4b-a4d0b524624d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:08.050073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1195/2388] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 13:05:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:05:08.062191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43e62f-320c-460b-9236-ff4c30c75ad2 service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:05:08.073387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43e62f-320c-460b-9236-ff4c30c75ad2 service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:05:08.078213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724c70b-5583-4d20-ac53-e0d3cf678318 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:05:08.078881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724c70b-5583-4d20-ac53-e0d3cf678318 service swift] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:08.079160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724c70b-5583-4d20-ac53-e0d3cf678318 service swift] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:08.079429 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724c70b-5583-4d20-ac53-e0d3cf678318 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:08.087525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724c70b-5583-4d20-ac53-e0d3cf678318 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:05:08.089087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724c70b-5583-4d20-ac53-e0d3cf678318 service swift] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:05:08.097486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1194/2389] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:05:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:05:09.901878 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aa364-1b98-4556-8c4b-a4d0b524624d None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:05:09.903698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2d4aa364-1b98-4556-8c4b-a4d0b524624d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:09.921319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aa364-1b98-4556-8c4b-a4d0b524624d None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:05:09.926935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ebffe-4d97-4c1d-b014-81143a25717f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:05:09.927589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ebffe-4d97-4c1d-b014-81143a25717f service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:09.927871 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ebffe-4d97-4c1d-b014-81143a25717f service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:09.928148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ebffe-4d97-4c1d-b014-81143a25717f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:09.929893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-dd1ebffe-4d97-4c1d-b014-81143a25717f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:10.008280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ebffe-4d97-4c1d-b014-81143a25717f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:05:10.009908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ebffe-4d97-4c1d-b014-81143a25717f service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:05:10.020503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1196/2390] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:05:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:05:18.093774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=94 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:05:20.015594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=51 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:05:23.775816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724c70b-5583-4d20-ac53-e0d3cf678318 service swift] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:05:23.785827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724c70b-5583-4d20-ac53-e0d3cf678318 service swift] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:05:23.789631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e83bd-e443-4ca9-9642-1805b1858672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:05:23.790294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e83bd-e443-4ca9-9642-1805b1858672 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:23.790624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e83bd-e443-4ca9-9642-1805b1858672 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:23.790902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e83bd-e443-4ca9-9642-1805b1858672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:23.792666 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f86e83bd-e443-4ca9-9642-1805b1858672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:23.802813 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e83bd-e443-4ca9-9642-1805b1858672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:05:23.804227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e83bd-e443-4ca9-9642-1805b1858672 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:05:23.812336 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1195/2391] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:05:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:05:33.808575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:05:34.525057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ebffe-4d97-4c1d-b014-81143a25717f service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:05:34.536223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ebffe-4d97-4c1d-b014-81143a25717f service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:05:34.541812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da5e95-b640-4dea-bee9-2388a5895c35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:05:34.542554 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da5e95-b640-4dea-bee9-2388a5895c35 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:34.542844 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da5e95-b640-4dea-bee9-2388a5895c35 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:34.543124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da5e95-b640-4dea-bee9-2388a5895c35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:34.544961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-91da5e95-b640-4dea-bee9-2388a5895c35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:34.603429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da5e95-b640-4dea-bee9-2388a5895c35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:05:34.605067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da5e95-b640-4dea-bee9-2388a5895c35 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:05:34.615622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1197/2392] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:05:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:05:36.518965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5cad3-0b1d-4006-9c64-6aa910ac2eff None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:36.519598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5cad3-0b1d-4006-9c64-6aa910ac2eff None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:36.519713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5cad3-0b1d-4006-9c64-6aa910ac2eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:36.539275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-46a5cad3-0b1d-4006-9c64-6aa910ac2eff None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:05:36.540458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-46a5cad3-0b1d-4006-9c64-6aa910ac2eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:36.547597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1196/2393] 10.222.0.45 () {58 vars in 967 bytes} [Tue May 24 13:05:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:05:36.561273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da5e95-b640-4dea-bee9-2388a5895c35 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:05:36.572645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da5e95-b640-4dea-bee9-2388a5895c35 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:05:36.578182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae139e-5762-4de2-822d-3c3ba5374f80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:05:36.578852 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae139e-5762-4de2-822d-3c3ba5374f80 service swift] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:36.579137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae139e-5762-4de2-822d-3c3ba5374f80 service swift] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:36.579414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae139e-5762-4de2-822d-3c3ba5374f80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:36.587832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae139e-5762-4de2-822d-3c3ba5374f80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:05:36.589443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae139e-5762-4de2-822d-3c3ba5374f80 service swift] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:05:36.597940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1198/2394] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:05:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:05:46.532965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:05:46.594144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=34 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:05:57.880058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87861d9-d0c9-443e-8ff9-1def340578ba None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:57.880589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87861d9-d0c9-443e-8ff9-1def340578ba None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:57.880589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87861d9-d0c9-443e-8ff9-1def340578ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:57.898058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-b87861d9-d0c9-443e-8ff9-1def340578ba None None] MFA Rules not processed for user `91a76f7af8a5467bbf567ca5b9db9ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:05:57.899269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b87861d9-d0c9-443e-8ff9-1def340578ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:57.905925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1197/2395] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:05:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:05:59.535335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae139e-5762-4de2-822d-3c3ba5374f80 service swift] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:05:59.536784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-14ae139e-5762-4de2-822d-3c3ba5374f80 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:59.552456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae139e-5762-4de2-822d-3c3ba5374f80 service swift] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:05:59.557949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a2bea-8e24-4976-85e0-b8d64a1e5027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:05:59.558559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a2bea-8e24-4976-85e0-b8d64a1e5027 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:05:59.558846 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a2bea-8e24-4976-85e0-b8d64a1e5027 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:05:59.559125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a2bea-8e24-4976-85e0-b8d64a1e5027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:05:59.560828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5b8a2bea-8e24-4976-85e0-b8d64a1e5027 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:05:59.618151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a2bea-8e24-4976-85e0-b8d64a1e5027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:05:59.619767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a2bea-8e24-4976-85e0-b8d64a1e5027 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:05:59.629241 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1199/2396] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:05:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:06:02.958518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87861d9-d0c9-443e-8ff9-1def340578ba None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:06:02.971739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87861d9-d0c9-443e-8ff9-1def340578ba None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:06:02.979669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38ffc7-4065-4991-b871-d6cd6f388a87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:06:02.979669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38ffc7-4065-4991-b871-d6cd6f388a87 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:06:02.979669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38ffc7-4065-4991-b871-d6cd6f388a87 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:06:02.979669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38ffc7-4065-4991-b871-d6cd6f388a87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:06:02.980931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9c38ffc7-4065-4991-b871-d6cd6f388a87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:06:02.993966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38ffc7-4065-4991-b871-d6cd6f388a87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:06:02.995613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38ffc7-4065-4991-b871-d6cd6f388a87 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:06:03.010125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1198/2397] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:06:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:06:09.624490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=28 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:06:13.002318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=9 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:06:40.045475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a2bea-8e24-4976-85e0-b8d64a1e5027 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:06:40.063597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a2bea-8e24-4976-85e0-b8d64a1e5027 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:06:40.070933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951a5e-0c5b-4035-aa8e-68b9f802c3ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:06:40.072215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951a5e-0c5b-4035-aa8e-68b9f802c3ff service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:06:40.072538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951a5e-0c5b-4035-aa8e-68b9f802c3ff service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:06:40.073067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951a5e-0c5b-4035-aa8e-68b9f802c3ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:06:40.075464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a7951a5e-0c5b-4035-aa8e-68b9f802c3ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:06:40.091882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7951a5e-0c5b-4035-aa8e-68b9f802c3ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:06:40.093585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7951a5e-0c5b-4035-aa8e-68b9f802c3ff service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:06:40.115717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1200/2398] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:06:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:06:42.901886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f483d-a1e7-43da-8c76-4c8e03e974c5 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:06:42.902427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f483d-a1e7-43da-8c76-4c8e03e974c5 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:06:42.902533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f483d-a1e7-43da-8c76-4c8e03e974c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:06:42.920046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-522f483d-a1e7-43da-8c76-4c8e03e974c5 None None] MFA Rules not processed for user `f7b0dfb3d6514817aba4ab34fd972ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:06:42.921248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-522f483d-a1e7-43da-8c76-4c8e03e974c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:06:42.928010 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1199/2399] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:06:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:06:43.234931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951a5e-0c5b-4035-aa8e-68b9f802c3ff service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:06:43.244983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951a5e-0c5b-4035-aa8e-68b9f802c3ff service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:06:43.248526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50698fb7-d1ae-4ee4-a051-4c8ae729e6f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:06:43.249163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50698fb7-d1ae-4ee4-a051-4c8ae729e6f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:06:43.249442 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50698fb7-d1ae-4ee4-a051-4c8ae729e6f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:06:43.249719 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50698fb7-d1ae-4ee4-a051-4c8ae729e6f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:06:43.251473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-50698fb7-d1ae-4ee4-a051-4c8ae729e6f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:06:43.260632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50698fb7-d1ae-4ee4-a051-4c8ae729e6f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:06:43.262053 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50698fb7-d1ae-4ee4-a051-4c8ae729e6f0 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:06:43.270201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1201/2400] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:06:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:06:50.629331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f483d-a1e7-43da-8c76-4c8e03e974c5 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:06:50.643962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f483d-a1e7-43da-8c76-4c8e03e974c5 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:06:50.649485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70628f1b-6d64-4080-9899-630016a11f0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:06:50.650152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628f1b-6d64-4080-9899-630016a11f0f service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:06:50.650498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628f1b-6d64-4080-9899-630016a11f0f service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:06:50.650805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628f1b-6d64-4080-9899-630016a11f0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:06:50.652674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-70628f1b-6d64-4080-9899-630016a11f0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:06:50.665566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70628f1b-6d64-4080-9899-630016a11f0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:06:50.667135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70628f1b-6d64-4080-9899-630016a11f0f service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:06:50.679331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1200/2401] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:06:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:06:53.266446 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=12 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:06:57.844534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0c5b2-f857-47b1-b79f-6d70ea237899 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:06:57.845088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0c5b2-f857-47b1-b79f-6d70ea237899 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:06:57.845088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0c5b2-f857-47b1-b79f-6d70ea237899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:06:57.863222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-23e0c5b2-f857-47b1-b79f-6d70ea237899 None None] MFA Rules not processed for user `d87d9358c42a43fea94927869c44dc82`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:06:57.864679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-23e0c5b2-f857-47b1-b79f-6d70ea237899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:06:57.918367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1202/2402] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:06:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:00.675660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=9 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:07.912620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=25 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:27.772611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70628f1b-6d64-4080-9899-630016a11f0f service nova] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:27.782021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70628f1b-6d64-4080-9899-630016a11f0f service nova] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:27.785580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388cd97-b289-44e1-9adf-347462d21a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:27.786225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388cd97-b289-44e1-9adf-347462d21a82 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:27.786534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388cd97-b289-44e1-9adf-347462d21a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:27.786806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388cd97-b289-44e1-9adf-347462d21a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:27.788568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4388cd97-b289-44e1-9adf-347462d21a82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:27.849953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388cd97-b289-44e1-9adf-347462d21a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:27.851422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388cd97-b289-44e1-9adf-347462d21a82 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:27.861020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1201/2403] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:07:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:07:30.595423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3a2be-f8ca-46a1-8996-80856fd43ed6 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:30.595946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3a2be-f8ca-46a1-8996-80856fd43ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:30.595946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3a2be-f8ca-46a1-8996-80856fd43ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:30.616150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7fb3a2be-f8ca-46a1-8996-80856fd43ed6 None None] MFA Rules not processed for user `cb3d46a7a0ae40e9b157b7199b18de27`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:30.617368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7fb3a2be-f8ca-46a1-8996-80856fd43ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:30.624344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1203/2404] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:31.494442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388cd97-b289-44e1-9adf-347462d21a82 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:31.535888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388cd97-b289-44e1-9adf-347462d21a82 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:31.546663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9777f-e74e-4993-b80c-97e6a161439b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:31.547349 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9777f-e74e-4993-b80c-97e6a161439b service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:31.547698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9777f-e74e-4993-b80c-97e6a161439b service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:31.548160 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9777f-e74e-4993-b80c-97e6a161439b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:31.549947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-53b9777f-e74e-4993-b80c-97e6a161439b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:31.569994 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3a2be-f8ca-46a1-8996-80856fd43ed6 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:31.570113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7fb3a2be-f8ca-46a1-8996-80856fd43ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:31.575669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9777f-e74e-4993-b80c-97e6a161439b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:31.577255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9777f-e74e-4993-b80c-97e6a161439b service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:31.589116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3a2be-f8ca-46a1-8996-80856fd43ed6 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:31.589507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1202/2405] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:07:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:07:31.609270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0776650-9482-4da9-b93d-297c90d5161a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:31.610792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0776650-9482-4da9-b93d-297c90d5161a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:31.611315 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0776650-9482-4da9-b93d-297c90d5161a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:31.611814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0776650-9482-4da9-b93d-297c90d5161a admin admin] PATH_INFO: `/v3/users/91a76f7af8a5467bbf567ca5b9db9ce0` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:31.613956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0776650-9482-4da9-b93d-297c90d5161a admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a76f7af8a5467bbf567ca5b9db9ce0)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:31.616496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0776650-9482-4da9-b93d-297c90d5161a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:31.665110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-f0776650-9482-4da9-b93d-297c90d5161a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a76f7af8a5467bbf567ca5b9db9ce0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:31.672901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-f0776650-9482-4da9-b93d-297c90d5161a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a76f7af8a5467bbf567ca5b9db9ce0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:31.673718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-f0776650-9482-4da9-b93d-297c90d5161a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a76f7af8a5467bbf567ca5b9db9ce0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:31.674435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-f0776650-9482-4da9-b93d-297c90d5161a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a76f7af8a5467bbf567ca5b9db9ce0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:31.687217 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-f0776650-9482-4da9-b93d-297c90d5161a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a76f7af8a5467bbf567ca5b9db9ce0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:31.700342 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1204/2406] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:07:31 2022] DELETE /identity/v3/users/91a76f7af8a5467bbf567ca5b9db9ce0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:31.893634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9777f-e74e-4993-b80c-97e6a161439b service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:31.906135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9777f-e74e-4993-b80c-97e6a161439b service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:31.977276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:31.978189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:31.978515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:31.978964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] PATH_INFO: `/v3/projects/a92196ee4d214c999d303c7f8e28a9ce` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:31.980262 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a92196ee4d214c999d303c7f8e28a9ce)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:31.982196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:32.039884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a92196ee4d214c999d303c7f8e28a9ce', 'request_id': 'req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:32.048989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a92196ee4d214c999d303c7f8e28a9ce', 'request_id': 'req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:32.054790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a92196ee4d214c999d303c7f8e28a9ce', 'request_id': 'req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:32.067530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] The token cache is being invalidate because project a92196ee4d214c999d303c7f8e28a9ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:07:32.067821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:32.068474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-09605e34-8dbe-40a1-8c81-9c68d7d7dfa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:32.070057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1203/2407] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:07:31 2022] DELETE /identity/v3/projects/a92196ee4d214c999d303c7f8e28a9ce => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:32.074878 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60ffe-2c82-4362-969f-dddb065b3d9a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.075163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60ffe-2c82-4362-969f-dddb065b3d9a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.075439 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60ffe-2c82-4362-969f-dddb065b3d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.092901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9fc60ffe-2c82-4362-969f-dddb065b3d9a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:32.094377 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9fc60ffe-2c82-4362-969f-dddb065b3d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:32.155362 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1205/2408] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:32.162978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7f8c6-dad4-4dbb-bda0-89bea1e950fa None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.163273 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7f8c6-dad4-4dbb-bda0-89bea1e950fa None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.163552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7f8c6-dad4-4dbb-bda0-89bea1e950fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.183935 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-b5f7f8c6-dad4-4dbb-bda0-89bea1e950fa None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:32.185257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b5f7f8c6-dad4-4dbb-bda0-89bea1e950fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:32.194201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1204/2409] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:32.200318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc60ffe-2c82-4362-969f-dddb065b3d9a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:32.209950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc60ffe-2c82-4362-969f-dddb065b3d9a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:32.214809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbd831-c087-4cf8-8db8-4c48628c85f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:32.215466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbd831-c087-4cf8-8db8-4c48628c85f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.215750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbd831-c087-4cf8-8db8-4c48628c85f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.216029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbd831-c087-4cf8-8db8-4c48628c85f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.216758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbd831-c087-4cf8-8db8-4c48628c85f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:07:32.217067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbd831-c087-4cf8-8db8-4c48628c85f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:32.218713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbd831-c087-4cf8-8db8-4c48628c85f5 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:32.230113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1206/2410] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:07:32.234418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7f8c6-dad4-4dbb-bda0-89bea1e950fa None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:32.240929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:32.244733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7f8c6-dad4-4dbb-bda0-89bea1e950fa None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:32.251203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778b08f-55c9-4262-b619-6e617623b0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:32.252015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778b08f-55c9-4262-b619-6e617623b0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.252478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778b08f-55c9-4262-b619-6e617623b0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.252921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778b08f-55c9-4262-b619-6e617623b0c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.253781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778b08f-55c9-4262-b619-6e617623b0c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:32.255813 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778b08f-55c9-4262-b619-6e617623b0c7 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:32.291439 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1205/2411] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:07:32 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:07:32.295430 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbd831-c087-4cf8-8db8-4c48628c85f5 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:32.305984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbd831-c087-4cf8-8db8-4c48628c85f5 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:32.351906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:32.362224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccdd03-0056-4643-bb73-bcf8848cd07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:32.363183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccdd03-0056-4643-bb73-bcf8848cd07d admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.363469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccdd03-0056-4643-bb73-bcf8848cd07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.363744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccdd03-0056-4643-bb73-bcf8848cd07d admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.364439 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccdd03-0056-4643-bb73-bcf8848cd07d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:32.366474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccdd03-0056-4643-bb73-bcf8848cd07d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:32.390070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1207/2412] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:07:32 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:07:32.394153 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778b08f-55c9-4262-b619-6e617623b0c7 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:32.405312 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778b08f-55c9-4262-b619-6e617623b0c7 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:32.411601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c40e02-50bb-498d-91e0-b0f1991eec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:32.412279 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c40e02-50bb-498d-91e0-b0f1991eec4b admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.412566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c40e02-50bb-498d-91e0-b0f1991eec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.412963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c40e02-50bb-498d-91e0-b0f1991eec4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.413695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c40e02-50bb-498d-91e0-b0f1991eec4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:32.415367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c40e02-50bb-498d-91e0-b0f1991eec4b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:32.426720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1206/2413] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:07:32.430755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccdd03-0056-4643-bb73-bcf8848cd07d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:32.440907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccdd03-0056-4643-bb73-bcf8848cd07d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:32.445553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2258b05-df24-4650-8088-391c807c6ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:32.446214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2258b05-df24-4650-8088-391c807c6ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.446539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2258b05-df24-4650-8088-391c807c6ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.446828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2258b05-df24-4650-8088-391c807c6ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.447553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2258b05-df24-4650-8088-391c807c6ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:32.449168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2258b05-df24-4650-8088-391c807c6ffe admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:32.459378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1208/2414] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:07:32.463504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c40e02-50bb-498d-91e0-b0f1991eec4b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:32.475825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c40e02-50bb-498d-91e0-b0f1991eec4b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:32.482662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f662f-2d95-49d0-8830-d8b680b37928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:32.483563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f662f-2d95-49d0-8830-d8b680b37928 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.483846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f662f-2d95-49d0-8830-d8b680b37928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.484164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f662f-2d95-49d0-8830-d8b680b37928 admin admin] PATH_INFO: `/v3/projects/0c817e41672848649bae61b0e22fbbe8/users/ff59c86f76eb4af5b21f40493cd3bfc9/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.501637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f662f-2d95-49d0-8830-d8b680b37928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c817e41672848649bae61b0e22fbbe8, user_id=ff59c86f76eb4af5b21f40493cd3bfc9, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:32.504362 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f662f-2d95-49d0-8830-d8b680b37928 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:32.519794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1207/2415] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:07:32 2022] PUT /identity/v3/projects/0c817e41672848649bae61b0e22fbbe8/users/ff59c86f76eb4af5b21f40493cd3bfc9/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:32.530695 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a659-1553-4128-8887-3e7f97e594e3 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.531102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a659-1553-4128-8887-3e7f97e594e3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.531489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a659-1553-4128-8887-3e7f97e594e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.570576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-1479a659-1553-4128-8887-3e7f97e594e3 None None] MFA Rules not processed for user `ff59c86f76eb4af5b21f40493cd3bfc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:32.571975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1479a659-1553-4128-8887-3e7f97e594e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:32.669451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1209/2416] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:32.670569 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f662f-2d95-49d0-8830-d8b680b37928 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:32.733509 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f662f-2d95-49d0-8830-d8b680b37928 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:32.739356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33363396-7885-4e02-ba77-ed5612ced1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:32.740050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33363396-7885-4e02-ba77-ed5612ced1eb admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.740389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33363396-7885-4e02-ba77-ed5612ced1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.740669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33363396-7885-4e02-ba77-ed5612ced1eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.741421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33363396-7885-4e02-ba77-ed5612ced1eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:32.743688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33363396-7885-4e02-ba77-ed5612ced1eb admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:32.770514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1208/2417] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:07:32 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:07:32.774395 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479a659-1553-4128-8887-3e7f97e594e3 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:32.786453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479a659-1553-4128-8887-3e7f97e594e3 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:32.843349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c8243-9f7a-4dc2-9e79-078d1782e5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:32.844806 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c8243-9f7a-4dc2-9e79-078d1782e5bd admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.845088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c8243-9f7a-4dc2-9e79-078d1782e5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.845362 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c8243-9f7a-4dc2-9e79-078d1782e5bd admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.846101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c8243-9f7a-4dc2-9e79-078d1782e5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:32.849184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c8243-9f7a-4dc2-9e79-078d1782e5bd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:32.872903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1210/2418] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:07:32 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:07:32.874098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33363396-7885-4e02-ba77-ed5612ced1eb admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:32.885434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33363396-7885-4e02-ba77-ed5612ced1eb admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:32.890029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de53a8c-d845-4150-a9d9-13ebf4b56f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:32.890729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de53a8c-d845-4150-a9d9-13ebf4b56f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.891535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de53a8c-d845-4150-a9d9-13ebf4b56f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.891806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de53a8c-d845-4150-a9d9-13ebf4b56f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.892526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de53a8c-d845-4150-a9d9-13ebf4b56f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:32.894117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de53a8c-d845-4150-a9d9-13ebf4b56f7d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:32.904685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1209/2419] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:07:32.909409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c8243-9f7a-4dc2-9e79-078d1782e5bd admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:32.919552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c8243-9f7a-4dc2-9e79-078d1782e5bd admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:32.924320 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3167a-760e-400d-abc5-ee04062c495f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:32.925234 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3167a-760e-400d-abc5-ee04062c495f admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:32.925519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3167a-760e-400d-abc5-ee04062c495f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:32.925798 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3167a-760e-400d-abc5-ee04062c495f admin admin] PATH_INFO: `/v3/projects/289dd7eaf0494873994286e629cc79ea/users/445d7cf1bd484fbf9aec241e488be96e/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:32.941348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3167a-760e-400d-abc5-ee04062c495f admin admin] RBAC: Authorizing `identity:create_grant(project_id=289dd7eaf0494873994286e629cc79ea, user_id=445d7cf1bd484fbf9aec241e488be96e, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:32.944081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3167a-760e-400d-abc5-ee04062c495f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:32.961770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1211/2420] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:07:32 2022] PUT /identity/v3/projects/289dd7eaf0494873994286e629cc79ea/users/445d7cf1bd484fbf9aec241e488be96e/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:07:32.965839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de53a8c-d845-4150-a9d9-13ebf4b56f7d admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:32.980021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de53a8c-d845-4150-a9d9-13ebf4b56f7d admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:33.049775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0f18d-6b1c-4129-8923-572fd196b785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:33.050404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0f18d-6b1c-4129-8923-572fd196b785 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:33.050698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0f18d-6b1c-4129-8923-572fd196b785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:33.050964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0f18d-6b1c-4129-8923-572fd196b785 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:33.051692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0f18d-6b1c-4129-8923-572fd196b785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:33.053293 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0f18d-6b1c-4129-8923-572fd196b785 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:33.065093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1210/2421] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:07:33.069275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3167a-760e-400d-abc5-ee04062c495f admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:33.080419 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3167a-760e-400d-abc5-ee04062c495f admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:33.085167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfb4de-9bad-41de-ac47-1e27937e25a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:33.086127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfb4de-9bad-41de-ac47-1e27937e25a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:33.086512 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfb4de-9bad-41de-ac47-1e27937e25a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:33.086858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfb4de-9bad-41de-ac47-1e27937e25a4 admin admin] PATH_INFO: `/v3/projects/289dd7eaf0494873994286e629cc79ea/users/445d7cf1bd484fbf9aec241e488be96e/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:33.088912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfb4de-9bad-41de-ac47-1e27937e25a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289dd7eaf0494873994286e629cc79ea, user_id=445d7cf1bd484fbf9aec241e488be96e, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:33.091687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfb4de-9bad-41de-ac47-1e27937e25a4 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:33.102084 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1212/2422] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:07:33 2022] PUT /identity/v3/projects/289dd7eaf0494873994286e629cc79ea/users/445d7cf1bd484fbf9aec241e488be96e/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:07:33.112029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9fa28-8929-4d50-97e4-33efcae3b4e4 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:33.112403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9fa28-8929-4d50-97e4-33efcae3b4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:33.112957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9fa28-8929-4d50-97e4-33efcae3b4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:33.133564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:33.145233 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-33a9fa28-8929-4d50-97e4-33efcae3b4e4 None None] MFA Rules not processed for user `445d7cf1bd484fbf9aec241e488be96e`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:33.146730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-33a9fa28-8929-4d50-97e4-33efcae3b4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:33.219613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1211/2423] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:33.231246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfb4de-9bad-41de-ac47-1e27937e25a4 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:33.233239 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-96dfb4de-9bad-41de-ac47-1e27937e25a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:33.249954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:33.309316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfb4de-9bad-41de-ac47-1e27937e25a4 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:33.313850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae1946-52b4-4f86-9d9a-949f0d566378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:33.314552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae1946-52b4-4f86-9d9a-949f0d566378 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:33.314843 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae1946-52b4-4f86-9d9a-949f0d566378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:33.315121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae1946-52b4-4f86-9d9a-949f0d566378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:33.325720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae1946-52b4-4f86-9d9a-949f0d566378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:33.327226 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae1946-52b4-4f86-9d9a-949f0d566378 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:33.338470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1213/2424] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:07:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:07:33.485194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9fa28-8929-4d50-97e4-33efcae3b4e4 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:33.550557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9fa28-8929-4d50-97e4-33efcae3b4e4 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:33.555228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:33.556121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:33.556408 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:33.556686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] PATH_INFO: `/v3/users/ff59c86f76eb4af5b21f40493cd3bfc9` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:33.557894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff59c86f76eb4af5b21f40493cd3bfc9)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:33.559916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:33.607033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff59c86f76eb4af5b21f40493cd3bfc9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:33.620975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff59c86f76eb4af5b21f40493cd3bfc9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:33.629750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff59c86f76eb4af5b21f40493cd3bfc9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:33.630424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff59c86f76eb4af5b21f40493cd3bfc9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:33.637969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff59c86f76eb4af5b21f40493cd3bfc9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:33.644329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1212/2425] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:07:33 2022] DELETE /identity/v3/users/ff59c86f76eb4af5b21f40493cd3bfc9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:07:33.648711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301bb0a-456c-4a09-9455-e9f488aeca85 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:33.649031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301bb0a-456c-4a09-9455-e9f488aeca85 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:33.649312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301bb0a-456c-4a09-9455-e9f488aeca85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:33.665975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-4301bb0a-456c-4a09-9455-e9f488aeca85 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:33.667204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4301bb0a-456c-4a09-9455-e9f488aeca85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:33.729550 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1214/2426] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:33.740746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:33.742782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:33.763032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:33.817559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96b67e-af5a-4850-a49c-a43e8a60b1b9 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:33.822093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388d287-b2cb-4f1c-bbbe-5cb7824fca58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:33.822898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388d287-b2cb-4f1c-bbbe-5cb7824fca58 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:33.823316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388d287-b2cb-4f1c-bbbe-5cb7824fca58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:33.823645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388d287-b2cb-4f1c-bbbe-5cb7824fca58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:33.834528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388d287-b2cb-4f1c-bbbe-5cb7824fca58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:33.836712 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388d287-b2cb-4f1c-bbbe-5cb7824fca58 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:33.848410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1213/2427] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:07:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:07:34.225434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301bb0a-456c-4a09-9455-e9f488aeca85 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:34.227334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4301bb0a-456c-4a09-9455-e9f488aeca85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:34.240248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301bb0a-456c-4a09-9455-e9f488aeca85 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:34.245768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:34.246739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:34.247023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:34.247298 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] PATH_INFO: `/v3/projects/0c817e41672848649bae61b0e22fbbe8` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:34.248531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c817e41672848649bae61b0e22fbbe8)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:34.250613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:34.305035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c817e41672848649bae61b0e22fbbe8', 'request_id': 'req-743029f8-76d8-4bf9-9208-72034172f923'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.312136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c817e41672848649bae61b0e22fbbe8', 'request_id': 'req-743029f8-76d8-4bf9-9208-72034172f923'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.318397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c817e41672848649bae61b0e22fbbe8', 'request_id': 'req-743029f8-76d8-4bf9-9208-72034172f923'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.329743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] The token cache is being invalidate because project 0c817e41672848649bae61b0e22fbbe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:07:34.329961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.330611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.332147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1215/2428] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:07:34 2022] DELETE /identity/v3/projects/0c817e41672848649bae61b0e22fbbe8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:34.336054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388d287-b2cb-4f1c-bbbe-5cb7824fca58 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:34.346369 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388d287-b2cb-4f1c-bbbe-5cb7824fca58 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:34.411835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8be33-4741-471b-b215-f7c72be5cb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:34.412740 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8be33-4741-471b-b215-f7c72be5cb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:34.413021 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8be33-4741-471b-b215-f7c72be5cb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:34.413300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8be33-4741-471b-b215-f7c72be5cb99 admin admin] PATH_INFO: `/v3/users/445d7cf1bd484fbf9aec241e488be96e` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:34.414505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8be33-4741-471b-b215-f7c72be5cb99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=445d7cf1bd484fbf9aec241e488be96e)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:34.416426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8be33-4741-471b-b215-f7c72be5cb99 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:34.462500 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6ea8be33-4741-471b-b215-f7c72be5cb99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '445d7cf1bd484fbf9aec241e488be96e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.476837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6ea8be33-4741-471b-b215-f7c72be5cb99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '445d7cf1bd484fbf9aec241e488be96e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.485046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6ea8be33-4741-471b-b215-f7c72be5cb99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '445d7cf1bd484fbf9aec241e488be96e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.485832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6ea8be33-4741-471b-b215-f7c72be5cb99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '445d7cf1bd484fbf9aec241e488be96e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.493567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6ea8be33-4741-471b-b215-f7c72be5cb99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '445d7cf1bd484fbf9aec241e488be96e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.499796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1214/2429] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:07:34 2022] DELETE /identity/v3/users/445d7cf1bd484fbf9aec241e488be96e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:34.763213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:34.764656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:34.778123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743029f8-76d8-4bf9-9208-72034172f923 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:34.817124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:34.837308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:34.838177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:34.838494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:34.838773 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] PATH_INFO: `/v3/projects/289dd7eaf0494873994286e629cc79ea` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:34.839952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] RBAC: Authorizing `identity:delete_project(project_id=289dd7eaf0494873994286e629cc79ea)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:34.841890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:34.886638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '289dd7eaf0494873994286e629cc79ea', 'request_id': 'req-5a466389-29ba-4de8-a2e1-e2841984e35e'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.894704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '289dd7eaf0494873994286e629cc79ea', 'request_id': 'req-5a466389-29ba-4de8-a2e1-e2841984e35e'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.900179 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '289dd7eaf0494873994286e629cc79ea', 'request_id': 'req-5a466389-29ba-4de8-a2e1-e2841984e35e'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.911310 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] The token cache is being invalidate because project 289dd7eaf0494873994286e629cc79ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:07:34.911590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.912208 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5a466389-29ba-4de8-a2e1-e2841984e35e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:34.913709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1216/2430] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:07:34 2022] DELETE /identity/v3/projects/289dd7eaf0494873994286e629cc79ea => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:34.919303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f3460-e8fb-49e1-af5d-66cdc90ad31d None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:34.919760 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f3460-e8fb-49e1-af5d-66cdc90ad31d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:34.919936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f3460-e8fb-49e1-af5d-66cdc90ad31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:34.938723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-8a6f3460-e8fb-49e1-af5d-66cdc90ad31d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:34.939907 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8a6f3460-e8fb-49e1-af5d-66cdc90ad31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:34.968882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:34.998893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1215/2431] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:35.007063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af614f-5ce4-4c69-b152-55e54da4013a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:35.007358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af614f-5ce4-4c69-b152-55e54da4013a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:35.007640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af614f-5ce4-4c69-b152-55e54da4013a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:35.025775 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-d1af614f-5ce4-4c69-b152-55e54da4013a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:35.027036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d1af614f-5ce4-4c69-b152-55e54da4013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:35.034829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1217/2432] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:35.038952 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f3460-e8fb-49e1-af5d-66cdc90ad31d None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:35.049149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f3460-e8fb-49e1-af5d-66cdc90ad31d None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:35.053741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96aedd-0aa0-4f9d-91d4-753c82df3206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:35.054464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96aedd-0aa0-4f9d-91d4-753c82df3206 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:35.054750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96aedd-0aa0-4f9d-91d4-753c82df3206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:35.055029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96aedd-0aa0-4f9d-91d4-753c82df3206 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:35.055761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96aedd-0aa0-4f9d-91d4-753c82df3206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:07:35.056066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96aedd-0aa0-4f9d-91d4-753c82df3206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:35.057653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96aedd-0aa0-4f9d-91d4-753c82df3206 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:35.073099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1216/2433] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:07:35.074180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af614f-5ce4-4c69-b152-55e54da4013a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:35.083353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af614f-5ce4-4c69-b152-55e54da4013a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:35.088051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267883c-1f05-4b5e-b1e3-42f2156e8635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:35.088710 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267883c-1f05-4b5e-b1e3-42f2156e8635 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:35.088992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267883c-1f05-4b5e-b1e3-42f2156e8635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:35.089267 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267883c-1f05-4b5e-b1e3-42f2156e8635 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:35.089965 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267883c-1f05-4b5e-b1e3-42f2156e8635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:35.091917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267883c-1f05-4b5e-b1e3-42f2156e8635 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:35.119201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1218/2434] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:07:35 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:07:35.123310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96aedd-0aa0-4f9d-91d4-753c82df3206 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:35.134046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96aedd-0aa0-4f9d-91d4-753c82df3206 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:35.189345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e1d7c-a1f1-40a6-a274-e9f26feb3a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:35.190011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e1d7c-a1f1-40a6-a274-e9f26feb3a85 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:35.190350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e1d7c-a1f1-40a6-a274-e9f26feb3a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:35.190628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e1d7c-a1f1-40a6-a274-e9f26feb3a85 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:35.191316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e1d7c-a1f1-40a6-a274-e9f26feb3a85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:35.193208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e1d7c-a1f1-40a6-a274-e9f26feb3a85 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:35.212524 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1217/2435] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:07:35 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:07:35.216717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267883c-1f05-4b5e-b1e3-42f2156e8635 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:35.227678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267883c-1f05-4b5e-b1e3-42f2156e8635 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:35.232216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3685-6282-4870-bb83-ec9c8bbd90cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:35.232872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3685-6282-4870-bb83-ec9c8bbd90cc admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:35.233153 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3685-6282-4870-bb83-ec9c8bbd90cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:35.233427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3685-6282-4870-bb83-ec9c8bbd90cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:35.234182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e3685-6282-4870-bb83-ec9c8bbd90cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:35.235839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e3685-6282-4870-bb83-ec9c8bbd90cc admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:35.246421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1219/2436] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:07:35.250018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e1d7c-a1f1-40a6-a274-e9f26feb3a85 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:35.259674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e1d7c-a1f1-40a6-a274-e9f26feb3a85 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:35.264221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851420c-e520-4ebb-9e94-edcd99109467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:35.264869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851420c-e520-4ebb-9e94-edcd99109467 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:35.265155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851420c-e520-4ebb-9e94-edcd99109467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:35.265434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851420c-e520-4ebb-9e94-edcd99109467 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:35.266685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851420c-e520-4ebb-9e94-edcd99109467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:35.268261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851420c-e520-4ebb-9e94-edcd99109467 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:35.277848 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1218/2437] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:07:35.282435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3685-6282-4870-bb83-ec9c8bbd90cc admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:35.292566 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3685-6282-4870-bb83-ec9c8bbd90cc admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:35.297104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbe250-d447-4b0c-bc98-2b808ea24a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:35.297998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbe250-d447-4b0c-bc98-2b808ea24a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:35.298299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbe250-d447-4b0c-bc98-2b808ea24a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:35.298587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbe250-d447-4b0c-bc98-2b808ea24a81 admin admin] PATH_INFO: `/v3/projects/d768f3406cf44d75be7bc62e4f56720a/users/7b4c25f28b0a4459915048ab11058388/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:35.313365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbe250-d447-4b0c-bc98-2b808ea24a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d768f3406cf44d75be7bc62e4f56720a, user_id=7b4c25f28b0a4459915048ab11058388, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:35.316062 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbe250-d447-4b0c-bc98-2b808ea24a81 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:35.329045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1220/2438] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:07:35 2022] PUT /identity/v3/projects/d768f3406cf44d75be7bc62e4f56720a/users/7b4c25f28b0a4459915048ab11058388/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:35.333739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43a8ce-2eb7-4fdd-a000-cb3853819469 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:35.334032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43a8ce-2eb7-4fdd-a000-cb3853819469 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:35.334341 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43a8ce-2eb7-4fdd-a000-cb3853819469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:35.351614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1a43a8ce-2eb7-4fdd-a000-cb3853819469 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:35.352801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1a43a8ce-2eb7-4fdd-a000-cb3853819469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:35.411684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1219/2439] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:35.421129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbe250-d447-4b0c-bc98-2b808ea24a81 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:35.422860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3dbbe250-d447-4b0c-bc98-2b808ea24a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:35.478670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbe250-d447-4b0c-bc98-2b808ea24a81 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:35.482331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11e1ae-c50a-46c7-b48c-8ce0dcab2dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:35.482977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11e1ae-c50a-46c7-b48c-8ce0dcab2dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:35.483265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11e1ae-c50a-46c7-b48c-8ce0dcab2dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:35.483543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11e1ae-c50a-46c7-b48c-8ce0dcab2dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:35.493170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11e1ae-c50a-46c7-b48c-8ce0dcab2dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:35.494669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11e1ae-c50a-46c7-b48c-8ce0dcab2dd3 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:35.503944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1221/2440] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:07:36.125979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf339d-9d8c-4b56-869f-4f9965ece760 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:36.126507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf339d-9d8c-4b56-869f-4f9965ece760 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:36.126716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf339d-9d8c-4b56-869f-4f9965ece760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:36.152658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-93bf339d-9d8c-4b56-869f-4f9965ece760 None None] MFA Rules not processed for user `7b4c25f28b0a4459915048ab11058388`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:36.153965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-93bf339d-9d8c-4b56-869f-4f9965ece760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:36.189867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:36.229945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1220/2441] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:36.234387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11e1ae-c50a-46c7-b48c-8ce0dcab2dd3 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:36.245492 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11e1ae-c50a-46c7-b48c-8ce0dcab2dd3 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:36.251608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36581bd4-7556-4e34-979c-02e4a29d87df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:36.252902 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36581bd4-7556-4e34-979c-02e4a29d87df admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:36.253289 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36581bd4-7556-4e34-979c-02e4a29d87df admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:36.253671 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36581bd4-7556-4e34-979c-02e4a29d87df admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:36.254518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36581bd4-7556-4e34-979c-02e4a29d87df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:36.256551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36581bd4-7556-4e34-979c-02e4a29d87df admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:36.261575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:36.294658 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1222/2442] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:07:36 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:07:36.299525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf339d-9d8c-4b56-869f-4f9965ece760 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:36.312391 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf339d-9d8c-4b56-869f-4f9965ece760 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:36.375365 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7add36-b87c-452f-a6f5-b735cda7e181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:36.376036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7add36-b87c-452f-a6f5-b735cda7e181 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:36.376320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7add36-b87c-452f-a6f5-b735cda7e181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:36.376592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7add36-b87c-452f-a6f5-b735cda7e181 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:36.377306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7add36-b87c-452f-a6f5-b735cda7e181 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:36.379243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7add36-b87c-452f-a6f5-b735cda7e181 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:36.400874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1221/2443] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:07:36 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:07:36.404758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36581bd4-7556-4e34-979c-02e4a29d87df admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:36.415609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36581bd4-7556-4e34-979c-02e4a29d87df admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:36.420303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dfa5c-b6a4-47c1-a80f-55888b01f046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:36.420962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dfa5c-b6a4-47c1-a80f-55888b01f046 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:36.421247 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dfa5c-b6a4-47c1-a80f-55888b01f046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:36.421521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dfa5c-b6a4-47c1-a80f-55888b01f046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:36.422803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dfa5c-b6a4-47c1-a80f-55888b01f046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:36.424412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dfa5c-b6a4-47c1-a80f-55888b01f046 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:36.434831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1223/2444] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:07:36.439433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7add36-b87c-452f-a6f5-b735cda7e181 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:36.449339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7add36-b87c-452f-a6f5-b735cda7e181 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:36.453960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c9975-8bd8-455e-b5bc-297fece1038e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:36.454858 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c9975-8bd8-455e-b5bc-297fece1038e admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:36.455142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c9975-8bd8-455e-b5bc-297fece1038e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:36.455416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c9975-8bd8-455e-b5bc-297fece1038e admin admin] PATH_INFO: `/v3/projects/8e28df4ac76c4792ba08262097653997/users/70b44a9e0ca8449cbd3e7823be994170/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:36.469686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c9975-8bd8-455e-b5bc-297fece1038e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e28df4ac76c4792ba08262097653997, user_id=70b44a9e0ca8449cbd3e7823be994170, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:36.472419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c9975-8bd8-455e-b5bc-297fece1038e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:36.485143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1222/2445] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:07:36 2022] PUT /identity/v3/projects/8e28df4ac76c4792ba08262097653997/users/70b44a9e0ca8449cbd3e7823be994170/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:36.489234 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dfa5c-b6a4-47c1-a80f-55888b01f046 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:36.499032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dfa5c-b6a4-47c1-a80f-55888b01f046 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:36.554230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f7f54-54e7-43ee-bb17-dc4500d4d744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:36.554857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f7f54-54e7-43ee-bb17-dc4500d4d744 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:36.555138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f7f54-54e7-43ee-bb17-dc4500d4d744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:36.555415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f7f54-54e7-43ee-bb17-dc4500d4d744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:36.556157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f7f54-54e7-43ee-bb17-dc4500d4d744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:36.557873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f7f54-54e7-43ee-bb17-dc4500d4d744 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:36.568088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1224/2446] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:07:36.572310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c9975-8bd8-455e-b5bc-297fece1038e admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:36.582531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c9975-8bd8-455e-b5bc-297fece1038e admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:36.587263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55960f-8590-4183-b6a2-de798ab87296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:36.588137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55960f-8590-4183-b6a2-de798ab87296 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:36.588422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55960f-8590-4183-b6a2-de798ab87296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:36.588699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55960f-8590-4183-b6a2-de798ab87296 admin admin] PATH_INFO: `/v3/projects/8e28df4ac76c4792ba08262097653997/users/70b44a9e0ca8449cbd3e7823be994170/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:36.590702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55960f-8590-4183-b6a2-de798ab87296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e28df4ac76c4792ba08262097653997, user_id=70b44a9e0ca8449cbd3e7823be994170, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:36.593340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55960f-8590-4183-b6a2-de798ab87296 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:36.603428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1223/2447] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:07:36 2022] PUT /identity/v3/projects/8e28df4ac76c4792ba08262097653997/users/70b44a9e0ca8449cbd3e7823be994170/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:07:37.835218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f70fb-44f3-47ac-9428-33ba445cbce9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:37.835724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f70fb-44f3-47ac-9428-33ba445cbce9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:37.835841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f70fb-44f3-47ac-9428-33ba445cbce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:37.861957 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ff9f70fb-44f3-47ac-9428-33ba445cbce9 None None] MFA Rules not processed for user `70b44a9e0ca8449cbd3e7823be994170`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:37.863477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ff9f70fb-44f3-47ac-9428-33ba445cbce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:37.935615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1225/2448] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:37.948065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55960f-8590-4183-b6a2-de798ab87296 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:37.949811 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5b55960f-8590-4183-b6a2-de798ab87296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:38.008143 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55960f-8590-4183-b6a2-de798ab87296 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:38.011746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65deea-95d7-465d-bb1e-1e26c1a2f299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:38.012395 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65deea-95d7-465d-bb1e-1e26c1a2f299 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:38.012681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65deea-95d7-465d-bb1e-1e26c1a2f299 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:38.012959 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65deea-95d7-465d-bb1e-1e26c1a2f299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:38.050008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:38.069280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65deea-95d7-465d-bb1e-1e26c1a2f299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:38.070774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65deea-95d7-465d-bb1e-1e26c1a2f299 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:38.079661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1224/2449] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:07:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:07:39.413487 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f70fb-44f3-47ac-9428-33ba445cbce9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:39.414875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ff9f70fb-44f3-47ac-9428-33ba445cbce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:39.427034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f70fb-44f3-47ac-9428-33ba445cbce9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:39.430513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e0dae-80d3-4857-8584-1d52e7066e3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:39.431147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e0dae-80d3-4857-8584-1d52e7066e3f service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:39.431425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e0dae-80d3-4857-8584-1d52e7066e3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:39.431698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e0dae-80d3-4857-8584-1d52e7066e3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:39.441028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e0dae-80d3-4857-8584-1d52e7066e3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:39.442488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e0dae-80d3-4857-8584-1d52e7066e3f service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:39.451863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1226/2450] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:07:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:07:48.075521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=10 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:49.447643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=79,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:51.757331 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65deea-95d7-465d-bb1e-1e26c1a2f299 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:51.758825 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9c65deea-95d7-465d-bb1e-1e26c1a2f299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:51.847170 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65deea-95d7-465d-bb1e-1e26c1a2f299 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:51.852192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241692fd-ad29-4fdc-8ab4-e83e628384e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:51.852847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241692fd-ad29-4fdc-8ab4-e83e628384e9 service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:51.853137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241692fd-ad29-4fdc-8ab4-e83e628384e9 service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:51.853419 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241692fd-ad29-4fdc-8ab4-e83e628384e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:51.855258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-241692fd-ad29-4fdc-8ab4-e83e628384e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:51.867778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241692fd-ad29-4fdc-8ab4-e83e628384e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:51.869367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241692fd-ad29-4fdc-8ab4-e83e628384e9 service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:51.879458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1225/2451] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:07:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:07:52.303736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e0dae-80d3-4857-8584-1d52e7066e3f service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:52.365282 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e0dae-80d3-4857-8584-1d52e7066e3f service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:52.369794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:52.370697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:52.370985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:52.371265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] PATH_INFO: `/v3/users/7b4c25f28b0a4459915048ab11058388` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:52.372451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b4c25f28b0a4459915048ab11058388)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:52.374432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:52.419674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b4c25f28b0a4459915048ab11058388'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.426833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b4c25f28b0a4459915048ab11058388'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.427453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b4c25f28b0a4459915048ab11058388'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.428058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b4c25f28b0a4459915048ab11058388'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.440065 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b4c25f28b0a4459915048ab11058388'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.452046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1227/2452] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:07:52 2022] DELETE /identity/v3/users/7b4c25f28b0a4459915048ab11058388 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:07:52.621932 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241692fd-ad29-4fdc-8ab4-e83e628384e9 service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:52.633384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241692fd-ad29-4fdc-8ab4-e83e628384e9 service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:52.689757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:52.690669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:52.690956 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:52.691235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] PATH_INFO: `/v3/projects/d768f3406cf44d75be7bc62e4f56720a` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:52.692429 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d768f3406cf44d75be7bc62e4f56720a)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:52.694394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:52.743625 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd768f3406cf44d75be7bc62e4f56720a', 'request_id': 'req-a8189e6f-291f-47f2-acf0-9ded91fac727'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.750645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd768f3406cf44d75be7bc62e4f56720a', 'request_id': 'req-a8189e6f-291f-47f2-acf0-9ded91fac727'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.756604 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd768f3406cf44d75be7bc62e4f56720a', 'request_id': 'req-a8189e6f-291f-47f2-acf0-9ded91fac727'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.767328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] The token cache is being invalidate because project d768f3406cf44d75be7bc62e4f56720a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:07:52.768447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.768875 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.770432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1226/2453] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:07:52 2022] DELETE /identity/v3/projects/d768f3406cf44d75be7bc62e4f56720a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:52.774394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:52.775826 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:52.792045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed1ef1-bb4f-46cd-9bf4-07b860f37cc8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:52.848734 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c0248-f375-45c6-8a19-f4b5e4cebc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:52.849714 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c0248-f375-45c6-8a19-f4b5e4cebc13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:52.850001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c0248-f375-45c6-8a19-f4b5e4cebc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:52.850275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c0248-f375-45c6-8a19-f4b5e4cebc13 admin admin] PATH_INFO: `/v3/users/70b44a9e0ca8449cbd3e7823be994170` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:52.851491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c0248-f375-45c6-8a19-f4b5e4cebc13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b44a9e0ca8449cbd3e7823be994170)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:52.853420 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c0248-f375-45c6-8a19-f4b5e4cebc13 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:52.875269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:52.899439 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c31c0248-f375-45c6-8a19-f4b5e4cebc13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b44a9e0ca8449cbd3e7823be994170'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.906443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c31c0248-f375-45c6-8a19-f4b5e4cebc13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b44a9e0ca8449cbd3e7823be994170'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.907063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c31c0248-f375-45c6-8a19-f4b5e4cebc13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b44a9e0ca8449cbd3e7823be994170'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.907674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c31c0248-f375-45c6-8a19-f4b5e4cebc13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b44a9e0ca8449cbd3e7823be994170'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.919379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c31c0248-f375-45c6-8a19-f4b5e4cebc13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b44a9e0ca8449cbd3e7823be994170'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:52.931794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1228/2454] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:07:52 2022] DELETE /identity/v3/users/70b44a9e0ca8449cbd3e7823be994170 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:53.099508 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:53.100943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:53.115309 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8189e6f-291f-47f2-acf0-9ded91fac727 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:53.134727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:53.175058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:53.175917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:53.176203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:53.176476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] PATH_INFO: `/v3/projects/8e28df4ac76c4792ba08262097653997` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:53.177640 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e28df4ac76c4792ba08262097653997)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:53.179550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:53.224870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e28df4ac76c4792ba08262097653997', 'request_id': 'req-cdf4520d-3d78-4181-bdbc-97619c5571e7'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:53.232988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e28df4ac76c4792ba08262097653997', 'request_id': 'req-cdf4520d-3d78-4181-bdbc-97619c5571e7'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:53.239399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e28df4ac76c4792ba08262097653997', 'request_id': 'req-cdf4520d-3d78-4181-bdbc-97619c5571e7'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:53.250032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] The token cache is being invalidate because project 8e28df4ac76c4792ba08262097653997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:07:53.250367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:53.251004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cdf4520d-3d78-4181-bdbc-97619c5571e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:07:53.252525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1227/2455] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:07:53 2022] DELETE /identity/v3/projects/8e28df4ac76c4792ba08262097653997 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:53.257319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4d13d-1256-4b2e-b651-8e10041db049 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:53.257605 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4d13d-1256-4b2e-b651-8e10041db049 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:53.257907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4d13d-1256-4b2e-b651-8e10041db049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:53.274975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-25a4d13d-1256-4b2e-b651-8e10041db049 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:53.276136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-25a4d13d-1256-4b2e-b651-8e10041db049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:53.334643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1229/2456] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:53.342239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d9176-e96e-4abb-ad88-1bf0eab759f8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:53.342593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d9176-e96e-4abb-ad88-1bf0eab759f8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:53.342880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d9176-e96e-4abb-ad88-1bf0eab759f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:53.361754 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-ab4d9176-e96e-4abb-ad88-1bf0eab759f8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:53.363254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ab4d9176-e96e-4abb-ad88-1bf0eab759f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:53.371044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1228/2457] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:53.374994 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4d13d-1256-4b2e-b651-8e10041db049 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:53.384640 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4d13d-1256-4b2e-b651-8e10041db049 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:53.389211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961d1eb-8bd6-44bf-b994-33836db3cdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:53.389865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d1eb-8bd6-44bf-b994-33836db3cdb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:53.390151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d1eb-8bd6-44bf-b994-33836db3cdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:53.390464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d1eb-8bd6-44bf-b994-33836db3cdb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:53.391199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961d1eb-8bd6-44bf-b994-33836db3cdb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:07:53.391509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961d1eb-8bd6-44bf-b994-33836db3cdb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:53.393118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961d1eb-8bd6-44bf-b994-33836db3cdb1 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:53.404626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1230/2458] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:07:53.409677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d9176-e96e-4abb-ad88-1bf0eab759f8 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:53.418806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d9176-e96e-4abb-ad88-1bf0eab759f8 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:53.423348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e9d37-ebcc-46d9-8985-17ca3fcd2833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:53.424519 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e9d37-ebcc-46d9-8985-17ca3fcd2833 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:53.424799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e9d37-ebcc-46d9-8985-17ca3fcd2833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:53.425072 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e9d37-ebcc-46d9-8985-17ca3fcd2833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:53.425748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e9d37-ebcc-46d9-8985-17ca3fcd2833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:53.427782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e9d37-ebcc-46d9-8985-17ca3fcd2833 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:53.451925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1229/2459] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:07:53 2022] POST /identity/v3/projects => generated 375 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:07:53.455943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961d1eb-8bd6-44bf-b994-33836db3cdb1 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:53.465742 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961d1eb-8bd6-44bf-b994-33836db3cdb1 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:53.520480 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00529a-5596-4148-9e34-42eb4e254517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:53.521139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00529a-5596-4148-9e34-42eb4e254517 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:53.521424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00529a-5596-4148-9e34-42eb4e254517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:53.521699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00529a-5596-4148-9e34-42eb4e254517 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:53.522436 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00529a-5596-4148-9e34-42eb4e254517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:53.524365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00529a-5596-4148-9e34-42eb4e254517 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:53.544286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1231/2460] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:07:53 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:07:53.548316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e9d37-ebcc-46d9-8985-17ca3fcd2833 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:53.559130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e9d37-ebcc-46d9-8985-17ca3fcd2833 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:53.563780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14bcc9-90a3-48cb-a747-73c4c533f2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:53.564475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14bcc9-90a3-48cb-a747-73c4c533f2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:53.564759 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14bcc9-90a3-48cb-a747-73c4c533f2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:53.565038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14bcc9-90a3-48cb-a747-73c4c533f2c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:53.565763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14bcc9-90a3-48cb-a747-73c4c533f2c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:53.568680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14bcc9-90a3-48cb-a747-73c4c533f2c6 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:53.581089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1230/2461] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:07:53.585025 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00529a-5596-4148-9e34-42eb4e254517 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:53.595021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00529a-5596-4148-9e34-42eb4e254517 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:53.600241 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2559e8b-5c03-40d6-b0b2-ae98cf9b0962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:53.600985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2559e8b-5c03-40d6-b0b2-ae98cf9b0962 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:53.601343 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2559e8b-5c03-40d6-b0b2-ae98cf9b0962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:53.601657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2559e8b-5c03-40d6-b0b2-ae98cf9b0962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:53.602417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2559e8b-5c03-40d6-b0b2-ae98cf9b0962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:53.604023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2559e8b-5c03-40d6-b0b2-ae98cf9b0962 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:53.614074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1232/2462] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:07:53.618012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14bcc9-90a3-48cb-a747-73c4c533f2c6 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:53.627894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14bcc9-90a3-48cb-a747-73c4c533f2c6 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:53.632520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ba3af-32a4-4c19-8990-0ffc470c58f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:53.633388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ba3af-32a4-4c19-8990-0ffc470c58f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:53.633673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ba3af-32a4-4c19-8990-0ffc470c58f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:53.633951 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ba3af-32a4-4c19-8990-0ffc470c58f5 admin admin] PATH_INFO: `/v3/projects/44c644a7b1344f6ea9bb1346a2cd6a67/users/4ebce1ed3b0046668163bab764155392/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:53.649896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ba3af-32a4-4c19-8990-0ffc470c58f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c644a7b1344f6ea9bb1346a2cd6a67, user_id=4ebce1ed3b0046668163bab764155392, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:53.652595 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ba3af-32a4-4c19-8990-0ffc470c58f5 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:53.666404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1231/2463] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:07:53 2022] PUT /identity/v3/projects/44c644a7b1344f6ea9bb1346a2cd6a67/users/4ebce1ed3b0046668163bab764155392/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:07:53.677416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dadb-f9fb-4b14-8ceb-3cbb0d614c2a None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:53.677711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dadb-f9fb-4b14-8ceb-3cbb0d614c2a None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:53.677993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dadb-f9fb-4b14-8ceb-3cbb0d614c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:53.684967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:53.705802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-4497dadb-f9fb-4b14-8ceb-3cbb0d614c2a None None] MFA Rules not processed for user `4ebce1ed3b0046668163bab764155392`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:07:53.707052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4497dadb-f9fb-4b14-8ceb-3cbb0d614c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:53.775878 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1233/2464] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:07:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:07:53.787219 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ba3af-32a4-4c19-8990-0ffc470c58f5 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:07:53.789396 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-cc1ba3af-32a4-4c19-8990-0ffc470c58f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:07:53.839336 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:07:53.844672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ba3af-32a4-4c19-8990-0ffc470c58f5 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:07:53.850395 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c5ea9-637b-4c40-b012-6dbb81ae3cb3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e47f1d65687e4cfcbce764076141702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:07:53.851207 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c5ea9-637b-4c40-b012-6dbb81ae3cb3 service glance] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:07:53.851705 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c5ea9-637b-4c40-b012-6dbb81ae3cb3 service glance] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:07:53.852176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c5ea9-637b-4c40-b012-6dbb81ae3cb3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:07:53.860423 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c5ea9-637b-4c40-b012-6dbb81ae3cb3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:07:53.861881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c5ea9-637b-4c40-b012-6dbb81ae3cb3 service glance] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:07:53.870759 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1232/2465] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:07:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:08:03.771340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=30 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:03.866804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:07.222358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497dadb-f9fb-4b14-8ceb-3cbb0d614c2a None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:07.225358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4497dadb-f9fb-4b14-8ceb-3cbb0d614c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:07.292171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497dadb-f9fb-4b14-8ceb-3cbb0d614c2a None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:07.297195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb89f0-19a8-484c-872d-9dabfb1c028d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:07.297837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb89f0-19a8-484c-872d-9dabfb1c028d service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:07.298121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb89f0-19a8-484c-872d-9dabfb1c028d service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:07.298434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb89f0-19a8-484c-872d-9dabfb1c028d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:07.300242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-eafb89f0-19a8-484c-872d-9dabfb1c028d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:07.311357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb89f0-19a8-484c-872d-9dabfb1c028d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:07.312940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb89f0-19a8-484c-872d-9dabfb1c028d service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:07.322627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1234/2466] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:08:15.572307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c5ea9-637b-4c40-b012-6dbb81ae3cb3 service glance] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:15.638498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c5ea9-637b-4c40-b012-6dbb81ae3cb3 service glance] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:15.647067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:15.647957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:15.648427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:15.648701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] PATH_INFO: `/v3/users/4ebce1ed3b0046668163bab764155392` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:15.649903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ebce1ed3b0046668163bab764155392)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:15.651880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:15.698097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ebce1ed3b0046668163bab764155392'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:15.709919 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ebce1ed3b0046668163bab764155392'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:15.717937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ebce1ed3b0046668163bab764155392'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:15.718576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ebce1ed3b0046668163bab764155392'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:15.726043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ebce1ed3b0046668163bab764155392'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:15.734888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1233/2467] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:08:15 2022] DELETE /identity/v3/users/4ebce1ed3b0046668163bab764155392 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:15.740179 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f0995-a505-4f8c-abd7-d6d8744bb0f9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:15.740691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f0995-a505-4f8c-abd7-d6d8744bb0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:15.740803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f0995-a505-4f8c-abd7-d6d8744bb0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:15.758727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-d85f0995-a505-4f8c-abd7-d6d8744bb0f9 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:15.759912 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d85f0995-a505-4f8c-abd7-d6d8744bb0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:15.822739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1235/2468] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:15.832195 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:15.833594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:15.904373 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84387604-e58c-4a70-b566-54878892ba83 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:15.908240 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77d269-0077-437f-a900-a1cfc40c3c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:15.909083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77d269-0077-437f-a900-a1cfc40c3c05 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:15.909397 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77d269-0077-437f-a900-a1cfc40c3c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:15.909683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77d269-0077-437f-a900-a1cfc40c3c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:15.922721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77d269-0077-437f-a900-a1cfc40c3c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:15.924176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77d269-0077-437f-a900-a1cfc40c3c05 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:15.934818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1234/2469] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:08:16.215135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f0995-a505-4f8c-abd7-d6d8744bb0f9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:16.216560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d85f0995-a505-4f8c-abd7-d6d8744bb0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:16.229552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f0995-a505-4f8c-abd7-d6d8744bb0f9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:16.234368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:16.235231 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:16.235513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:16.235788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] PATH_INFO: `/v3/projects/44c644a7b1344f6ea9bb1346a2cd6a67` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:16.237243 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c644a7b1344f6ea9bb1346a2cd6a67)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:16.239253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:16.286061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c644a7b1344f6ea9bb1346a2cd6a67', 'request_id': 'req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:16.293420 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c644a7b1344f6ea9bb1346a2cd6a67', 'request_id': 'req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:16.299440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c644a7b1344f6ea9bb1346a2cd6a67', 'request_id': 'req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:16.310383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] The token cache is being invalidate because project 44c644a7b1344f6ea9bb1346a2cd6a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:08:16.310672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:16.311302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3a3a85b7-76fe-4d3c-9c94-7b5ffc3eabfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:16.312839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1236/2470] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:08:16 2022] DELETE /identity/v3/projects/44c644a7b1344f6ea9bb1346a2cd6a67 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:16.317829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616e8b2-c1e6-4646-a2ca-ff1a0e9220a3 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:16.318115 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616e8b2-c1e6-4646-a2ca-ff1a0e9220a3 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:16.318439 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616e8b2-c1e6-4646-a2ca-ff1a0e9220a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:16.335964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-8616e8b2-c1e6-4646-a2ca-ff1a0e9220a3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:16.337216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8616e8b2-c1e6-4646-a2ca-ff1a0e9220a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:16.356093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:16.403068 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1235/2471] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:16.411551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0badc-f21a-4ef6-82f3-465b59d1a3b3 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:16.411853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0badc-f21a-4ef6-82f3-465b59d1a3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:16.412140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0badc-f21a-4ef6-82f3-465b59d1a3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:16.430720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-73f0badc-f21a-4ef6-82f3-465b59d1a3b3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:16.431953 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-73f0badc-f21a-4ef6-82f3-465b59d1a3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:16.440136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1237/2472] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:16.444982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616e8b2-c1e6-4646-a2ca-ff1a0e9220a3 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:16.454967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616e8b2-c1e6-4646-a2ca-ff1a0e9220a3 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:16.460123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cee05-dee9-4975-8144-bb8385ae8995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:16.460885 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cee05-dee9-4975-8144-bb8385ae8995 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:16.461168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cee05-dee9-4975-8144-bb8385ae8995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:16.461445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cee05-dee9-4975-8144-bb8385ae8995 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:16.462175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cee05-dee9-4975-8144-bb8385ae8995 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:08:16.462517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cee05-dee9-4975-8144-bb8385ae8995 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:16.464123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cee05-dee9-4975-8144-bb8385ae8995 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:16.476738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1236/2473] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:08:16.480948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0badc-f21a-4ef6-82f3-465b59d1a3b3 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:16.491016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0badc-f21a-4ef6-82f3-465b59d1a3b3 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:16.495840 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8ae52-f950-40f9-80e0-7a958d22a946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:16.496502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ae52-f950-40f9-80e0-7a958d22a946 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:16.496786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ae52-f950-40f9-80e0-7a958d22a946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:16.497092 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ae52-f950-40f9-80e0-7a958d22a946 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:16.497802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8ae52-f950-40f9-80e0-7a958d22a946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:16.499771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8ae52-f950-40f9-80e0-7a958d22a946 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:16.503243 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:16.527718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1238/2474] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:08:16 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:08:16.531899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cee05-dee9-4975-8144-bb8385ae8995 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:16.542029 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cee05-dee9-4975-8144-bb8385ae8995 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:16.597481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b390c9-1afa-4499-9231-36d7acaab749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:16.598117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b390c9-1afa-4499-9231-36d7acaab749 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:16.598505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b390c9-1afa-4499-9231-36d7acaab749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:16.598789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b390c9-1afa-4499-9231-36d7acaab749 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:16.599493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b390c9-1afa-4499-9231-36d7acaab749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:16.601413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b390c9-1afa-4499-9231-36d7acaab749 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:16.621108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1237/2475] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:08:16 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:08:16.625119 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8ae52-f950-40f9-80e0-7a958d22a946 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:16.635787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8ae52-f950-40f9-80e0-7a958d22a946 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:16.640424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93424caf-e6ab-4f15-ad7d-3234048551e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:16.641078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93424caf-e6ab-4f15-ad7d-3234048551e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:16.641367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93424caf-e6ab-4f15-ad7d-3234048551e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:16.641647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93424caf-e6ab-4f15-ad7d-3234048551e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:16.642415 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93424caf-e6ab-4f15-ad7d-3234048551e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:16.644062 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93424caf-e6ab-4f15-ad7d-3234048551e1 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:16.654513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1239/2476] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:08:16.658084 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b390c9-1afa-4499-9231-36d7acaab749 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:16.668219 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b390c9-1afa-4499-9231-36d7acaab749 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:16.672922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f2337-6b13-4eb0-af92-e6bc363779a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:16.673578 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f2337-6b13-4eb0-af92-e6bc363779a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:16.673867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f2337-6b13-4eb0-af92-e6bc363779a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:16.674150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f2337-6b13-4eb0-af92-e6bc363779a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:16.674915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f2337-6b13-4eb0-af92-e6bc363779a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:16.676534 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f2337-6b13-4eb0-af92-e6bc363779a7 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:16.686366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1238/2477] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:08:16.690331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93424caf-e6ab-4f15-ad7d-3234048551e1 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:16.701198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93424caf-e6ab-4f15-ad7d-3234048551e1 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:16.705798 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49d816-2918-4d8d-908f-58b6c15be603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:16.707358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49d816-2918-4d8d-908f-58b6c15be603 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:16.707645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49d816-2918-4d8d-908f-58b6c15be603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:16.707924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49d816-2918-4d8d-908f-58b6c15be603 admin admin] PATH_INFO: `/v3/projects/834ff3efbebd40749616b579b857378a/users/d589764e4cb14f68bca13a0ebae66106/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:16.723779 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49d816-2918-4d8d-908f-58b6c15be603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834ff3efbebd40749616b579b857378a, user_id=d589764e4cb14f68bca13a0ebae66106, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:16.726526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49d816-2918-4d8d-908f-58b6c15be603 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:16.739198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1240/2478] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:08:16 2022] PUT /identity/v3/projects/834ff3efbebd40749616b579b857378a/users/d589764e4cb14f68bca13a0ebae66106/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:16.750318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be3b06-0c5d-46b5-8e2b-ac25ea56e1c4 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:16.750626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be3b06-0c5d-46b5-8e2b-ac25ea56e1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:16.750906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be3b06-0c5d-46b5-8e2b-ac25ea56e1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:16.778170 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-22be3b06-0c5d-46b5-8e2b-ac25ea56e1c4 None None] MFA Rules not processed for user `d589764e4cb14f68bca13a0ebae66106`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:16.779401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-22be3b06-0c5d-46b5-8e2b-ac25ea56e1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:16.841759 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1239/2479] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:16.846109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49d816-2918-4d8d-908f-58b6c15be603 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:16.906983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49d816-2918-4d8d-908f-58b6c15be603 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:16.911890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24b988-b17e-4655-8602-e729d8f0e26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:16.912572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24b988-b17e-4655-8602-e729d8f0e26a admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:16.912859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24b988-b17e-4655-8602-e729d8f0e26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:16.913134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24b988-b17e-4655-8602-e729d8f0e26a admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:16.913818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24b988-b17e-4655-8602-e729d8f0e26a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:16.915788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24b988-b17e-4655-8602-e729d8f0e26a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:16.940446 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1241/2480] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:08:16 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:08:16.945124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be3b06-0c5d-46b5-8e2b-ac25ea56e1c4 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:16.956148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be3b06-0c5d-46b5-8e2b-ac25ea56e1c4 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:16.968452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:17.014352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ff781-b80e-46f8-bc29-774340763bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:17.015015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ff781-b80e-46f8-bc29-774340763bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.015299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ff781-b80e-46f8-bc29-774340763bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.015572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ff781-b80e-46f8-bc29-774340763bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.016244 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ff781-b80e-46f8-bc29-774340763bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:17.018147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ff781-b80e-46f8-bc29-774340763bf4 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:17.037642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1240/2481] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:08:16 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:08:17.041683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24b988-b17e-4655-8602-e729d8f0e26a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:17.052494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24b988-b17e-4655-8602-e729d8f0e26a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:17.057107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6483c0a-633c-440d-a8a2-7b6dd18dfcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:17.057761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6483c0a-633c-440d-a8a2-7b6dd18dfcb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.058049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6483c0a-633c-440d-a8a2-7b6dd18dfcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.058366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6483c0a-633c-440d-a8a2-7b6dd18dfcb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.059090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6483c0a-633c-440d-a8a2-7b6dd18dfcb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:17.060699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6483c0a-633c-440d-a8a2-7b6dd18dfcb8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:17.071686 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1242/2482] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:08:17.075581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ff781-b80e-46f8-bc29-774340763bf4 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:17.085624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ff781-b80e-46f8-bc29-774340763bf4 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:17.090229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844ca97-71c3-490c-8057-cc28e7e88176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:17.090914 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844ca97-71c3-490c-8057-cc28e7e88176 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.091193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844ca97-71c3-490c-8057-cc28e7e88176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.091468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844ca97-71c3-490c-8057-cc28e7e88176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.092191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844ca97-71c3-490c-8057-cc28e7e88176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:17.093766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844ca97-71c3-490c-8057-cc28e7e88176 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:17.103518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1241/2483] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:08:17.107469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6483c0a-633c-440d-a8a2-7b6dd18dfcb8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:17.118081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6483c0a-633c-440d-a8a2-7b6dd18dfcb8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:17.122726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f071e-acc7-4108-8044-2dbe7049ba2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:17.123599 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f071e-acc7-4108-8044-2dbe7049ba2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.123888 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f071e-acc7-4108-8044-2dbe7049ba2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.124175 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f071e-acc7-4108-8044-2dbe7049ba2d admin admin] PATH_INFO: `/v3/projects/b5b44e639a214413856c5722067f381b/users/ffff1aef160548c6ba3fdbacfcc08f93/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.139018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f071e-acc7-4108-8044-2dbe7049ba2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b44e639a214413856c5722067f381b, user_id=ffff1aef160548c6ba3fdbacfcc08f93, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:17.141702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f071e-acc7-4108-8044-2dbe7049ba2d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:17.154352 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1243/2484] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:08:17 2022] PUT /identity/v3/projects/b5b44e639a214413856c5722067f381b/users/ffff1aef160548c6ba3fdbacfcc08f93/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:17.164733 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31005b15-2f25-494c-bc74-e712c3915ca2 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.164964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31005b15-2f25-494c-bc74-e712c3915ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.165245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31005b15-2f25-494c-bc74-e712c3915ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.191272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-31005b15-2f25-494c-bc74-e712c3915ca2 None None] MFA Rules not processed for user `ffff1aef160548c6ba3fdbacfcc08f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:17.192458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-31005b15-2f25-494c-bc74-e712c3915ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:17.255533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1242/2485] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:17.260161 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f071e-acc7-4108-8044-2dbe7049ba2d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:17.321854 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f071e-acc7-4108-8044-2dbe7049ba2d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:17.326410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81f52-cd34-4825-8e0d-07d219c4d75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:17.327057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81f52-cd34-4825-8e0d-07d219c4d75d admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.327339 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81f52-cd34-4825-8e0d-07d219c4d75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.327608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81f52-cd34-4825-8e0d-07d219c4d75d admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.328363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81f52-cd34-4825-8e0d-07d219c4d75d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:17.330274 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81f52-cd34-4825-8e0d-07d219c4d75d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:17.333393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:17.355931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1244/2486] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:08:17 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:08:17.360283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31005b15-2f25-494c-bc74-e712c3915ca2 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:17.371792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31005b15-2f25-494c-bc74-e712c3915ca2 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:17.428467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02adf2e-d8d6-443a-ad26-95eb8d86aa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:17.429073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02adf2e-d8d6-443a-ad26-95eb8d86aa8a admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.429361 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02adf2e-d8d6-443a-ad26-95eb8d86aa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.429641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02adf2e-d8d6-443a-ad26-95eb8d86aa8a admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.430365 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02adf2e-d8d6-443a-ad26-95eb8d86aa8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:17.432281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02adf2e-d8d6-443a-ad26-95eb8d86aa8a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:17.451799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1243/2487] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:08:17 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:08:17.456067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81f52-cd34-4825-8e0d-07d219c4d75d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:17.466228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81f52-cd34-4825-8e0d-07d219c4d75d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:17.471079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561950d-e3af-4e28-a73e-c5a6391cd393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:17.471740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561950d-e3af-4e28-a73e-c5a6391cd393 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.472023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561950d-e3af-4e28-a73e-c5a6391cd393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.472930 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561950d-e3af-4e28-a73e-c5a6391cd393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.473685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561950d-e3af-4e28-a73e-c5a6391cd393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:17.475338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561950d-e3af-4e28-a73e-c5a6391cd393 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:17.485887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1245/2488] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:08:17.491012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02adf2e-d8d6-443a-ad26-95eb8d86aa8a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:17.500792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02adf2e-d8d6-443a-ad26-95eb8d86aa8a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:17.505449 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca91414-9edd-4839-8ab0-d7dedf26c169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:17.506357 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca91414-9edd-4839-8ab0-d7dedf26c169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.506642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca91414-9edd-4839-8ab0-d7dedf26c169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.506916 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca91414-9edd-4839-8ab0-d7dedf26c169 admin admin] PATH_INFO: `/v3/projects/a075bd56d1534c18960a6f284a6d94ec/users/e6ef9bc2798d47058a405f7aab1d3b6b/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.518767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:17.522827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca91414-9edd-4839-8ab0-d7dedf26c169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a075bd56d1534c18960a6f284a6d94ec, user_id=e6ef9bc2798d47058a405f7aab1d3b6b, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:17.525881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca91414-9edd-4839-8ab0-d7dedf26c169 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:17.540354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1244/2489] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:08:17 2022] PUT /identity/v3/projects/a075bd56d1534c18960a6f284a6d94ec/users/e6ef9bc2798d47058a405f7aab1d3b6b/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:17.544324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561950d-e3af-4e28-a73e-c5a6391cd393 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:17.554236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561950d-e3af-4e28-a73e-c5a6391cd393 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:17.617628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea251f87-3889-4501-b3b1-2b143474a1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:17.618324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251f87-3889-4501-b3b1-2b143474a1ca admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.618616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251f87-3889-4501-b3b1-2b143474a1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.618898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251f87-3889-4501-b3b1-2b143474a1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.619627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea251f87-3889-4501-b3b1-2b143474a1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:17.621237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea251f87-3889-4501-b3b1-2b143474a1ca admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:17.631236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1246/2490] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:08:17.635780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca91414-9edd-4839-8ab0-d7dedf26c169 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:17.646817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca91414-9edd-4839-8ab0-d7dedf26c169 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:17.651478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737b348-3ff2-448c-82f9-c37de8a85f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:17.652375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737b348-3ff2-448c-82f9-c37de8a85f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.652661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737b348-3ff2-448c-82f9-c37de8a85f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.652957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737b348-3ff2-448c-82f9-c37de8a85f24 admin admin] PATH_INFO: `/v3/projects/a075bd56d1534c18960a6f284a6d94ec/users/e6ef9bc2798d47058a405f7aab1d3b6b/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.654937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737b348-3ff2-448c-82f9-c37de8a85f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a075bd56d1534c18960a6f284a6d94ec, user_id=e6ef9bc2798d47058a405f7aab1d3b6b, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:17.657585 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737b348-3ff2-448c-82f9-c37de8a85f24 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:17.668732 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1245/2491] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:08:17 2022] PUT /identity/v3/projects/a075bd56d1534c18960a6f284a6d94ec/users/e6ef9bc2798d47058a405f7aab1d3b6b/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:17.678853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bdded-4b0d-44bf-8ed3-f25ba86e6e35 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.679151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bdded-4b0d-44bf-8ed3-f25ba86e6e35 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.679500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bdded-4b0d-44bf-8ed3-f25ba86e6e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.704845 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-aa5bdded-4b0d-44bf-8ed3-f25ba86e6e35 None None] MFA Rules not processed for user `e6ef9bc2798d47058a405f7aab1d3b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:17.706044 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-aa5bdded-4b0d-44bf-8ed3-f25ba86e6e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:17.772451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1247/2492] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:17.783730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737b348-3ff2-448c-82f9-c37de8a85f24 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:17.786055 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4737b348-3ff2-448c-82f9-c37de8a85f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:17.841317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737b348-3ff2-448c-82f9-c37de8a85f24 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:17.844911 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847e68d-8e0e-42bf-855c-dde32828f2b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:17.845548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847e68d-8e0e-42bf-855c-dde32828f2b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:17.845833 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847e68d-8e0e-42bf-855c-dde32828f2b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:17.846113 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847e68d-8e0e-42bf-855c-dde32828f2b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:17.902847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847e68d-8e0e-42bf-855c-dde32828f2b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:17.902847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847e68d-8e0e-42bf-855c-dde32828f2b1 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:17.915446 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1246/2493] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:08:19.388368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bdded-4b0d-44bf-8ed3-f25ba86e6e35 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:19.389752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-aa5bdded-4b0d-44bf-8ed3-f25ba86e6e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:19.401977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bdded-4b0d-44bf-8ed3-f25ba86e6e35 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:19.405732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522c763-7560-4cdc-8af1-861fc104a780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:19.406404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522c763-7560-4cdc-8af1-861fc104a780 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:19.407009 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522c763-7560-4cdc-8af1-861fc104a780 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:19.407009 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522c763-7560-4cdc-8af1-861fc104a780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:19.460701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:19.465547 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522c763-7560-4cdc-8af1-861fc104a780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:19.467477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522c763-7560-4cdc-8af1-861fc104a780 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:19.476408 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1248/2494] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:08:20.008387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847e68d-8e0e-42bf-855c-dde32828f2b1 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:20.016886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847e68d-8e0e-42bf-855c-dde32828f2b1 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:20.020595 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bf69b-ff22-403f-be19-84734e38463b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:20.021240 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bf69b-ff22-403f-be19-84734e38463b service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:20.021522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bf69b-ff22-403f-be19-84734e38463b service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:20.021798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bf69b-ff22-403f-be19-84734e38463b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:20.030652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980bf69b-ff22-403f-be19-84734e38463b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:20.032066 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980bf69b-ff22-403f-be19-84734e38463b service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:20.044510 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1247/2495] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:08:23.879771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522c763-7560-4cdc-8af1-861fc104a780 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:23.940445 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522c763-7560-4cdc-8af1-861fc104a780 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:23.944991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:23.945850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:23.946138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:23.946451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] PATH_INFO: `/v3/users/d589764e4cb14f68bca13a0ebae66106` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:23.947623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d589764e4cb14f68bca13a0ebae66106)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:23.949555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:23.994531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd589764e4cb14f68bca13a0ebae66106'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.001699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd589764e4cb14f68bca13a0ebae66106'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.002259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd589764e4cb14f68bca13a0ebae66106'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.002901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd589764e4cb14f68bca13a0ebae66106'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.014713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd589764e4cb14f68bca13a0ebae66106'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.026473 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1249/2496] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:08:23 2022] DELETE /identity/v3/users/d589764e4cb14f68bca13a0ebae66106 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:24.030860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d3441-32e0-4cad-8f13-dd78bfd38678 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:24.031310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d3441-32e0-4cad-8f13-dd78bfd38678 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:24.031417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d3441-32e0-4cad-8f13-dd78bfd38678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:24.048660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-d48d3441-32e0-4cad-8f13-dd78bfd38678 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:24.049925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d48d3441-32e0-4cad-8f13-dd78bfd38678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:24.107784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1248/2497] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:24.118297 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:24.119683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:24.174962 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f22460-f9ff-47fb-afd4-5808851161cb admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:24.179156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42763d55-cf40-427d-b78e-e27a98f9d5aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:24.179791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42763d55-cf40-427d-b78e-e27a98f9d5aa service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:24.180071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42763d55-cf40-427d-b78e-e27a98f9d5aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:24.180344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42763d55-cf40-427d-b78e-e27a98f9d5aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:24.190464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42763d55-cf40-427d-b78e-e27a98f9d5aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:24.191942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42763d55-cf40-427d-b78e-e27a98f9d5aa service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:24.201466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1250/2498] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:08:24.466059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d3441-32e0-4cad-8f13-dd78bfd38678 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:24.467768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d48d3441-32e0-4cad-8f13-dd78bfd38678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:24.480819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d3441-32e0-4cad-8f13-dd78bfd38678 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:24.485478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:24.486434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:24.487088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:24.487375 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] PATH_INFO: `/v3/projects/834ff3efbebd40749616b579b857378a` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:24.489061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834ff3efbebd40749616b579b857378a)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:24.491136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:24.508002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:24.541008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834ff3efbebd40749616b579b857378a', 'request_id': 'req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.548731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834ff3efbebd40749616b579b857378a', 'request_id': 'req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.556283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834ff3efbebd40749616b579b857378a', 'request_id': 'req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.567996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] The token cache is being invalidate because project 834ff3efbebd40749616b579b857378a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:08:24.568286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.568989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.571188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1249/2499] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:08:24 2022] DELETE /identity/v3/projects/834ff3efbebd40749616b579b857378a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:24.574974 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42763d55-cf40-427d-b78e-e27a98f9d5aa service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:24.584645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42763d55-cf40-427d-b78e-e27a98f9d5aa service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:24.597330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:24.648126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:24.649002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:24.649288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:24.649562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] PATH_INFO: `/v3/users/ffff1aef160548c6ba3fdbacfcc08f93` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:24.650792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffff1aef160548c6ba3fdbacfcc08f93)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:24.652791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:24.705269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffff1aef160548c6ba3fdbacfcc08f93'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.708764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffff1aef160548c6ba3fdbacfcc08f93'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.709542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffff1aef160548c6ba3fdbacfcc08f93'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.710198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffff1aef160548c6ba3fdbacfcc08f93'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.722042 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffff1aef160548c6ba3fdbacfcc08f93'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:24.734574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1251/2500] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:08:24 2022] DELETE /identity/v3/users/ffff1aef160548c6ba3fdbacfcc08f93 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:08:25.030265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:25.031790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:25.045635 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8bf5cf-4ac0-49da-b45d-fb74c116ac62 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:25.105008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:25.105884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:25.106166 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:25.106473 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] PATH_INFO: `/v3/projects/b5b44e639a214413856c5722067f381b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:25.107683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5b44e639a214413856c5722067f381b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:25.109599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:25.156897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5b44e639a214413856c5722067f381b', 'request_id': 'req-fe2da229-e8bc-492a-b78b-f81d25d5b800'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.165142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5b44e639a214413856c5722067f381b', 'request_id': 'req-fe2da229-e8bc-492a-b78b-f81d25d5b800'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.171074 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5b44e639a214413856c5722067f381b', 'request_id': 'req-fe2da229-e8bc-492a-b78b-f81d25d5b800'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.183692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] The token cache is being invalidate because project b5b44e639a214413856c5722067f381b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:08:25.184032 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.184730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.186349 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1250/2501] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:08:25 2022] DELETE /identity/v3/projects/b5b44e639a214413856c5722067f381b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:25.190312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:25.192362 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:25.204819 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28d989-f9f7-4013-9584-46fe3f70e56b admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:25.266344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07ec91-cd36-47a7-b86d-6f1a6c6e721a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:25.267232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07ec91-cd36-47a7-b86d-6f1a6c6e721a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:25.267518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07ec91-cd36-47a7-b86d-6f1a6c6e721a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:25.267799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07ec91-cd36-47a7-b86d-6f1a6c6e721a admin admin] PATH_INFO: `/v3/users/e6ef9bc2798d47058a405f7aab1d3b6b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:25.268992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07ec91-cd36-47a7-b86d-6f1a6c6e721a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ef9bc2798d47058a405f7aab1d3b6b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:25.271004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07ec91-cd36-47a7-b86d-6f1a6c6e721a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:25.317384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:25.328061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9e07ec91-cd36-47a7-b86d-6f1a6c6e721a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ef9bc2798d47058a405f7aab1d3b6b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.336224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9e07ec91-cd36-47a7-b86d-6f1a6c6e721a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ef9bc2798d47058a405f7aab1d3b6b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.337145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9e07ec91-cd36-47a7-b86d-6f1a6c6e721a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ef9bc2798d47058a405f7aab1d3b6b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.337754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9e07ec91-cd36-47a7-b86d-6f1a6c6e721a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ef9bc2798d47058a405f7aab1d3b6b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.350365 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9e07ec91-cd36-47a7-b86d-6f1a6c6e721a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ef9bc2798d47058a405f7aab1d3b6b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.363525 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1252/2502] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:08:25 2022] DELETE /identity/v3/users/e6ef9bc2798d47058a405f7aab1d3b6b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:08:25.630065 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:25.631588 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:25.644701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2da229-e8bc-492a-b78b-f81d25d5b800 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:25.677790 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:25.705302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:25.706159 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:25.707819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:25.707819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] PATH_INFO: `/v3/projects/a075bd56d1534c18960a6f284a6d94ec` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:25.708022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a075bd56d1534c18960a6f284a6d94ec)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:25.709792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:25.756558 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a075bd56d1534c18960a6f284a6d94ec', 'request_id': 'req-684e625e-22b6-4923-9b6b-a30ae72c8c27'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.766765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a075bd56d1534c18960a6f284a6d94ec', 'request_id': 'req-684e625e-22b6-4923-9b6b-a30ae72c8c27'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.773431 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a075bd56d1534c18960a6f284a6d94ec', 'request_id': 'req-684e625e-22b6-4923-9b6b-a30ae72c8c27'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.785598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] The token cache is being invalidate because project a075bd56d1534c18960a6f284a6d94ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:08:25.785886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.786579 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-684e625e-22b6-4923-9b6b-a30ae72c8c27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:25.788335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1251/2503] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:08:25 2022] DELETE /identity/v3/projects/a075bd56d1534c18960a6f284a6d94ec => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:25.792935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165605f-d842-48fd-bce3-00f9696465f3 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:25.793221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165605f-d842-48fd-bce3-00f9696465f3 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:25.793496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165605f-d842-48fd-bce3-00f9696465f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:25.811855 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-1165605f-d842-48fd-bce3-00f9696465f3 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:25.813073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1165605f-d842-48fd-bce3-00f9696465f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:25.872339 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1253/2504] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:25.880224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09743c26-632a-40b7-9d98-d92cffab548b None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:25.880451 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09743c26-632a-40b7-9d98-d92cffab548b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:25.880763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09743c26-632a-40b7-9d98-d92cffab548b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:25.899019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-09743c26-632a-40b7-9d98-d92cffab548b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:25.900199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-09743c26-632a-40b7-9d98-d92cffab548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:25.907923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1252/2505] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:25.912522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165605f-d842-48fd-bce3-00f9696465f3 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:25.921923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165605f-d842-48fd-bce3-00f9696465f3 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:25.926465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0125d1-e6b0-400a-a319-6bd056e32c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:25.927110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0125d1-e6b0-400a-a319-6bd056e32c8f admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:25.927392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0125d1-e6b0-400a-a319-6bd056e32c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:25.927666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0125d1-e6b0-400a-a319-6bd056e32c8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:25.928382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0125d1-e6b0-400a-a319-6bd056e32c8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:08:25.928684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0125d1-e6b0-400a-a319-6bd056e32c8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:25.930303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0125d1-e6b0-400a-a319-6bd056e32c8f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:25.941666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1254/2506] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:08:25.945872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09743c26-632a-40b7-9d98-d92cffab548b None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:25.954899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09743c26-632a-40b7-9d98-d92cffab548b None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:25.959917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298e810-df64-4ba0-b4d1-dc6bcd975ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:25.960601 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298e810-df64-4ba0-b4d1-dc6bcd975ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:25.960886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298e810-df64-4ba0-b4d1-dc6bcd975ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:25.961164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298e810-df64-4ba0-b4d1-dc6bcd975ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:25.963574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298e810-df64-4ba0-b4d1-dc6bcd975ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:25.963772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298e810-df64-4ba0-b4d1-dc6bcd975ae4 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:25.992389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1253/2507] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:08:25 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:08:25.996567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0125d1-e6b0-400a-a319-6bd056e32c8f admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:26.006579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0125d1-e6b0-400a-a319-6bd056e32c8f admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:26.062468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852331c-66cf-4472-b311-a3b1896d9699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:26.063136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852331c-66cf-4472-b311-a3b1896d9699 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:26.063427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852331c-66cf-4472-b311-a3b1896d9699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:26.063707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852331c-66cf-4472-b311-a3b1896d9699 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:26.064409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852331c-66cf-4472-b311-a3b1896d9699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:26.066450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852331c-66cf-4472-b311-a3b1896d9699 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:26.086433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1255/2508] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:08:25 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:08:26.090562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298e810-df64-4ba0-b4d1-dc6bcd975ae4 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:26.101479 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298e810-df64-4ba0-b4d1-dc6bcd975ae4 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:26.106073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff430d-6735-438f-bfcb-c1b6d57967bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:26.106763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff430d-6735-438f-bfcb-c1b6d57967bd admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:26.107048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff430d-6735-438f-bfcb-c1b6d57967bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:26.107324 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff430d-6735-438f-bfcb-c1b6d57967bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:26.108054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff430d-6735-438f-bfcb-c1b6d57967bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:26.109654 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff430d-6735-438f-bfcb-c1b6d57967bd admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:26.120113 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1254/2509] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:08:26.123831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852331c-66cf-4472-b311-a3b1896d9699 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:26.133598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852331c-66cf-4472-b311-a3b1896d9699 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:26.138102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a650e0-6729-407e-ba53-450a93558d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:26.138792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a650e0-6729-407e-ba53-450a93558d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:26.139079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a650e0-6729-407e-ba53-450a93558d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:26.139359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a650e0-6729-407e-ba53-450a93558d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:26.140076 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a650e0-6729-407e-ba53-450a93558d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:26.141675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a650e0-6729-407e-ba53-450a93558d37 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:26.151913 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1256/2510] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:08:26.155684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff430d-6735-438f-bfcb-c1b6d57967bd admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:26.165974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff430d-6735-438f-bfcb-c1b6d57967bd admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:26.170817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00518591-624d-424f-8359-bbe3085b285a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:26.171722 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518591-624d-424f-8359-bbe3085b285a admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:26.172013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518591-624d-424f-8359-bbe3085b285a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:26.172296 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518591-624d-424f-8359-bbe3085b285a admin admin] PATH_INFO: `/v3/projects/2c6af87b80bf42b0a3cf39ac6bbcada4/users/7be8601d679446abb88244be346a4656/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:26.187957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00518591-624d-424f-8359-bbe3085b285a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6af87b80bf42b0a3cf39ac6bbcada4, user_id=7be8601d679446abb88244be346a4656, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:26.190709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00518591-624d-424f-8359-bbe3085b285a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:26.203629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1255/2511] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:08:26 2022] PUT /identity/v3/projects/2c6af87b80bf42b0a3cf39ac6bbcada4/users/7be8601d679446abb88244be346a4656/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:26.214425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b05fa-a0df-48bb-8dd4-24771c6d7b51 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:26.214923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b05fa-a0df-48bb-8dd4-24771c6d7b51 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:26.215017 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b05fa-a0df-48bb-8dd4-24771c6d7b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:26.241265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-281b05fa-a0df-48bb-8dd4-24771c6d7b51 None None] MFA Rules not processed for user `7be8601d679446abb88244be346a4656`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:26.242511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-281b05fa-a0df-48bb-8dd4-24771c6d7b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:26.250474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:26.306514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1257/2512] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:26.318108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00518591-624d-424f-8359-bbe3085b285a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:26.319855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-00518591-624d-424f-8359-bbe3085b285a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:26.375267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00518591-624d-424f-8359-bbe3085b285a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:26.379440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd9f60-fdec-4819-a4ac-cb203515fd84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:26.380081 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd9f60-fdec-4819-a4ac-cb203515fd84 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:26.380362 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd9f60-fdec-4819-a4ac-cb203515fd84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:26.380642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd9f60-fdec-4819-a4ac-cb203515fd84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:26.389119 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd9f60-fdec-4819-a4ac-cb203515fd84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:26.390512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd9f60-fdec-4819-a4ac-cb203515fd84 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:26.399381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1256/2513] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:08:29.966638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b05fa-a0df-48bb-8dd4-24771c6d7b51 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:30.029929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b05fa-a0df-48bb-8dd4-24771c6d7b51 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:30.044471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc7fff-c841-40ba-9856-098140cdee8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:30.045850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc7fff-c841-40ba-9856-098140cdee8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:30.046137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc7fff-c841-40ba-9856-098140cdee8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:30.046462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc7fff-c841-40ba-9856-098140cdee8c admin admin] PATH_INFO: `/v3/users/7be8601d679446abb88244be346a4656` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:30.047983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc7fff-c841-40ba-9856-098140cdee8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be8601d679446abb88244be346a4656)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:30.049943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc7fff-c841-40ba-9856-098140cdee8c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:30.100230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-d3dc7fff-c841-40ba-9856-098140cdee8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be8601d679446abb88244be346a4656'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:30.107708 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-d3dc7fff-c841-40ba-9856-098140cdee8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be8601d679446abb88244be346a4656'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:30.108975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-d3dc7fff-c841-40ba-9856-098140cdee8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be8601d679446abb88244be346a4656'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:30.109540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-d3dc7fff-c841-40ba-9856-098140cdee8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be8601d679446abb88244be346a4656'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:30.122849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-d3dc7fff-c841-40ba-9856-098140cdee8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be8601d679446abb88244be346a4656'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:30.135680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1258/2514] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:08:29 2022] DELETE /identity/v3/users/7be8601d679446abb88244be346a4656 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:30.140151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb6787-9bfe-4462-a1c0-8862f749b8e8 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:30.140693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb6787-9bfe-4462-a1c0-8862f749b8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:30.140793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb6787-9bfe-4462-a1c0-8862f749b8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:30.147943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:30.162634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-b5cb6787-9bfe-4462-a1c0-8862f749b8e8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:30.164117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b5cb6787-9bfe-4462-a1c0-8862f749b8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:30.204968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c105d0f-9a2c-4dec-bcb0-c1875d0d90b2 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:30.205355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c105d0f-9a2c-4dec-bcb0-c1875d0d90b2 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:30.205694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c105d0f-9a2c-4dec-bcb0-c1875d0d90b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:30.225665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-1c105d0f-9a2c-4dec-bcb0-c1875d0d90b2 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:30.226894 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1c105d0f-9a2c-4dec-bcb0-c1875d0d90b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:30.228165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1257/2515] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:30.238049 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb6787-9bfe-4462-a1c0-8862f749b8e8 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:30.239356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b5cb6787-9bfe-4462-a1c0-8862f749b8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:30.252059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:30.288236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1259/2516] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 13:08:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:08:30.296603 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb6787-9bfe-4462-a1c0-8862f749b8e8 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:30.302944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ec9d8-1cb6-4ed0-af68-b3335069694d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:30.303715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ec9d8-1cb6-4ed0-af68-b3335069694d service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:30.304009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ec9d8-1cb6-4ed0-af68-b3335069694d service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:30.304292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ec9d8-1cb6-4ed0-af68-b3335069694d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:30.314978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ec9d8-1cb6-4ed0-af68-b3335069694d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:30.316536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ec9d8-1cb6-4ed0-af68-b3335069694d service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:30.326589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1258/2517] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:08:30.339407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c105d0f-9a2c-4dec-bcb0-c1875d0d90b2 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:30.340802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1c105d0f-9a2c-4dec-bcb0-c1875d0d90b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:30.443531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c105d0f-9a2c-4dec-bcb0-c1875d0d90b2 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:30.450431 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4541b-03f7-4679-84aa-7db1ae4a8320 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:30.451344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4541b-03f7-4679-84aa-7db1ae4a8320 service swift] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:30.451723 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4541b-03f7-4679-84aa-7db1ae4a8320 service swift] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:30.452094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4541b-03f7-4679-84aa-7db1ae4a8320 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:30.463465 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4541b-03f7-4679-84aa-7db1ae4a8320 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:30.465743 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4541b-03f7-4679-84aa-7db1ae4a8320 service swift] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:30.475078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1260/2518] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:08:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 13:08:30.653646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ec9d8-1cb6-4ed0-af68-b3335069694d service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:30.655179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b07ec9d8-1cb6-4ed0-af68-b3335069694d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:30.670735 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ec9d8-1cb6-4ed0-af68-b3335069694d service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:30.675681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:30.676715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:30.677161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:30.677510 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] PATH_INFO: `/v3/projects/2c6af87b80bf42b0a3cf39ac6bbcada4` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:30.678818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6af87b80bf42b0a3cf39ac6bbcada4)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:30.680876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:30.741139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6af87b80bf42b0a3cf39ac6bbcada4', 'request_id': 'req-52fee7e3-d68a-469d-8523-99dea2d31aac'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:30.749089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6af87b80bf42b0a3cf39ac6bbcada4', 'request_id': 'req-52fee7e3-d68a-469d-8523-99dea2d31aac'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:30.755973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6af87b80bf42b0a3cf39ac6bbcada4', 'request_id': 'req-52fee7e3-d68a-469d-8523-99dea2d31aac'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:30.768464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] The token cache is being invalidate because project 2c6af87b80bf42b0a3cf39ac6bbcada4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:08:30.768907 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:30.770276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-52fee7e3-d68a-469d-8523-99dea2d31aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:08:30.772210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1259/2519] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:08:30 2022] DELETE /identity/v3/projects/2c6af87b80bf42b0a3cf39ac6bbcada4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:30.776984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d137d5f-b039-4689-bab3-1947a09b9306 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:30.778464 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d137d5f-b039-4689-bab3-1947a09b9306 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:30.778959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d137d5f-b039-4689-bab3-1947a09b9306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:30.799190 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-0d137d5f-b039-4689-bab3-1947a09b9306 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:30.800406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0d137d5f-b039-4689-bab3-1947a09b9306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:30.870024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1261/2520] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:30.879395 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108b0cb-42ef-435d-9f44-63073570562a None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:30.879894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108b0cb-42ef-435d-9f44-63073570562a None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:30.880177 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108b0cb-42ef-435d-9f44-63073570562a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:30.902517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-3108b0cb-42ef-435d-9f44-63073570562a None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:30.903886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3108b0cb-42ef-435d-9f44-63073570562a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:30.913409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1260/2521] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:30.918586 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d137d5f-b039-4689-bab3-1947a09b9306 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:30.930429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d137d5f-b039-4689-bab3-1947a09b9306 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:30.935708 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8dbff-466b-429c-b39a-5756961c6f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:30.936421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8dbff-466b-429c-b39a-5756961c6f69 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:30.936740 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8dbff-466b-429c-b39a-5756961c6f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:30.937655 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8dbff-466b-429c-b39a-5756961c6f69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:30.938450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8dbff-466b-429c-b39a-5756961c6f69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:08:30.938769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8dbff-466b-429c-b39a-5756961c6f69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:30.940425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8dbff-466b-429c-b39a-5756961c6f69 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:30.968709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1262/2522] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:08:30.973854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108b0cb-42ef-435d-9f44-63073570562a None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:30.985518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108b0cb-42ef-435d-9f44-63073570562a None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:30.991888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595a5b0-6800-4b25-a92c-e8b647854675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:30.992799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a5b0-6800-4b25-a92c-e8b647854675 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:30.993215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a5b0-6800-4b25-a92c-e8b647854675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:30.993955 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a5b0-6800-4b25-a92c-e8b647854675 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:30.995834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595a5b0-6800-4b25-a92c-e8b647854675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:30.998516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595a5b0-6800-4b25-a92c-e8b647854675 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:31.024974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:31.037372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1261/2523] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:08:30 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:08:31.038946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8dbff-466b-429c-b39a-5756961c6f69 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:31.051200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8dbff-466b-429c-b39a-5756961c6f69 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:31.116366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:31.131299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfab933-50e2-467c-a13e-cdcb8ebb7d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:31.132001 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfab933-50e2-467c-a13e-cdcb8ebb7d26 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:31.132432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfab933-50e2-467c-a13e-cdcb8ebb7d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:31.132718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfab933-50e2-467c-a13e-cdcb8ebb7d26 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:31.133425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfab933-50e2-467c-a13e-cdcb8ebb7d26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:31.135417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfab933-50e2-467c-a13e-cdcb8ebb7d26 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:31.160371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1263/2524] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:08:31 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:08:31.164137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595a5b0-6800-4b25-a92c-e8b647854675 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:31.177461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595a5b0-6800-4b25-a92c-e8b647854675 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:31.183503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb317c11-304a-4d10-b8af-3d6a5d00508f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:31.184285 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb317c11-304a-4d10-b8af-3d6a5d00508f admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:31.184642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb317c11-304a-4d10-b8af-3d6a5d00508f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:31.185038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb317c11-304a-4d10-b8af-3d6a5d00508f admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:31.185849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb317c11-304a-4d10-b8af-3d6a5d00508f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:31.187577 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb317c11-304a-4d10-b8af-3d6a5d00508f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:31.201298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1262/2525] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:08:31.205426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfab933-50e2-467c-a13e-cdcb8ebb7d26 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:31.216643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfab933-50e2-467c-a13e-cdcb8ebb7d26 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:31.221531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a95b6-7cb5-48fc-a5f4-8a2b8c7fbcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:31.222242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a95b6-7cb5-48fc-a5f4-8a2b8c7fbcfd admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:31.222589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a95b6-7cb5-48fc-a5f4-8a2b8c7fbcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:31.222842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a95b6-7cb5-48fc-a5f4-8a2b8c7fbcfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:31.223713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a95b6-7cb5-48fc-a5f4-8a2b8c7fbcfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:31.225351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a95b6-7cb5-48fc-a5f4-8a2b8c7fbcfd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:31.238313 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1264/2526] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:08:31.242092 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb317c11-304a-4d10-b8af-3d6a5d00508f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:31.254866 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb317c11-304a-4d10-b8af-3d6a5d00508f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:31.260087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908bd02-7607-4dff-b316-ef16abbda150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:31.261012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908bd02-7607-4dff-b316-ef16abbda150 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:31.261304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908bd02-7607-4dff-b316-ef16abbda150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:31.261584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908bd02-7607-4dff-b316-ef16abbda150 admin admin] PATH_INFO: `/v3/projects/51cfc9ec9dcc49c0b16229698640b951/users/b70413462fee46769f9c3f20b1d6881d/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:31.282611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908bd02-7607-4dff-b316-ef16abbda150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51cfc9ec9dcc49c0b16229698640b951, user_id=b70413462fee46769f9c3f20b1d6881d, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:31.285731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908bd02-7607-4dff-b316-ef16abbda150 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:31.303313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1263/2527] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:08:31 2022] PUT /identity/v3/projects/51cfc9ec9dcc49c0b16229698640b951/users/b70413462fee46769f9c3f20b1d6881d/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:08:31.313127 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450e47c-6a43-4466-b2ef-70c98f4c0f53 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:31.313788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450e47c-6a43-4466-b2ef-70c98f4c0f53 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:31.314243 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450e47c-6a43-4466-b2ef-70c98f4c0f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:31.349074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a450e47c-6a43-4466-b2ef-70c98f4c0f53 None None] MFA Rules not processed for user `b70413462fee46769f9c3f20b1d6881d`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:31.350670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a450e47c-6a43-4466-b2ef-70c98f4c0f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:31.441670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1265/2528] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:31.449320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908bd02-7607-4dff-b316-ef16abbda150 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:31.451280 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9908bd02-7607-4dff-b316-ef16abbda150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:31.533639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908bd02-7607-4dff-b316-ef16abbda150 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:31.538714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd23e1-82b3-4a12-85f1-0f78e2f48bd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:31.539403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd23e1-82b3-4a12-85f1-0f78e2f48bd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:31.539817 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd23e1-82b3-4a12-85f1-0f78e2f48bd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:31.540112 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd23e1-82b3-4a12-85f1-0f78e2f48bd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:31.552850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd23e1-82b3-4a12-85f1-0f78e2f48bd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:31.554401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd23e1-82b3-4a12-85f1-0f78e2f48bd5 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:31.564692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1264/2529] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:08:33.022030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e183c9-bf77-4e56-b44e-c729b5f5e1c9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:33.022590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e183c9-bf77-4e56-b44e-c729b5f5e1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:33.022704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e183c9-bf77-4e56-b44e-c729b5f5e1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:33.041428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-82e183c9-bf77-4e56-b44e-c729b5f5e1c9 None None] MFA Rules not processed for user `b70413462fee46769f9c3f20b1d6881d`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:33.043111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-82e183c9-bf77-4e56-b44e-c729b5f5e1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:33.051390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1266/2530] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:08:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:41.560476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:43.035404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:45.844043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec64e61-83d5-4476-94c9-db7b36f345ab None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:45.844613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec64e61-83d5-4476-94c9-db7b36f345ab None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:45.844713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec64e61-83d5-4476-94c9-db7b36f345ab None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:45.846083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1265/2531] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 13:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:08:45.852806 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d49d2f-e6de-4470-9747-731eb2faafce None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:45.853275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d49d2f-e6de-4470-9747-731eb2faafce None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:45.853392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d49d2f-e6de-4470-9747-731eb2faafce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:45.873167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-c7d49d2f-e6de-4470-9747-731eb2faafce None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:45.874322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c7d49d2f-e6de-4470-9747-731eb2faafce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:45.945109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1267/2532] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 13:08:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:45.957033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec64e61-83d5-4476-94c9-db7b36f345ab None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:45.958891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3ec64e61-83d5-4476-94c9-db7b36f345ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:46.020123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec64e61-83d5-4476-94c9-db7b36f345ab None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:46.023652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce79155-3c04-431a-b553-1ac5764a4085 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e47f1d65687e4cfcbce764076141702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:46.024304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79155-3c04-431a-b553-1ac5764a4085 service glance] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:46.024593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79155-3c04-431a-b553-1ac5764a4085 service glance] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:46.024873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79155-3c04-431a-b553-1ac5764a4085 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:46.035394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce79155-3c04-431a-b553-1ac5764a4085 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:46.036822 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce79155-3c04-431a-b553-1ac5764a4085 service glance] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:46.047343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1266/2533] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:08:46.113435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbe4b1-a38b-4876-94a3-88b0de3628d5 None None] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:46.113760 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbe4b1-a38b-4876-94a3-88b0de3628d5 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:46.114046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbe4b1-a38b-4876-94a3-88b0de3628d5 None None] PATH_INFO: `/` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:46.115477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1268/2534] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 13:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:08:46.120051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8282b-46b4-4d07-a90b-410157ccc24c None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:46.120347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8282b-46b4-4d07-a90b-410157ccc24c None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:46.120660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8282b-46b4-4d07-a90b-410157ccc24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:46.139058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-52d8282b-46b4-4d07-a90b-410157ccc24c None None] MFA Rules not processed for user `bb111be6bc2946d3b50b8d99a89cf8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:46.140184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-52d8282b-46b4-4d07-a90b-410157ccc24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:46.191464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1267/2535] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 13:08:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:46.202935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbe4b1-a38b-4876-94a3-88b0de3628d5 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:46.204639 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bbfbe4b1-a38b-4876-94a3-88b0de3628d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:46.218032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbe4b1-a38b-4876-94a3-88b0de3628d5 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:46.221487 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932d5b3-98a3-45ea-96c2-ed6960d1a855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:46.222784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d5b3-98a3-45ea-96c2-ed6960d1a855 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:46.223069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d5b3-98a3-45ea-96c2-ed6960d1a855 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:46.223340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d5b3-98a3-45ea-96c2-ed6960d1a855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:46.230219 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932d5b3-98a3-45ea-96c2-ed6960d1a855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:46.231697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932d5b3-98a3-45ea-96c2-ed6960d1a855 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:46.239605 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1269/2536] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:08:46.320317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a3656-b32b-4393-9c36-bf1bdf4a8496 None None] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:46.320611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a3656-b32b-4393-9c36-bf1bdf4a8496 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:46.320896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a3656-b32b-4393-9c36-bf1bdf4a8496 None None] PATH_INFO: `/` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:46.322367 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1268/2537] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 13:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:08:46.327026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea6a36-69e7-4b8f-ad9f-0e61be41b22f None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:46.327317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea6a36-69e7-4b8f-ad9f-0e61be41b22f None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:46.327622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea6a36-69e7-4b8f-ad9f-0e61be41b22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:46.346662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-54ea6a36-69e7-4b8f-ad9f-0e61be41b22f None None] MFA Rules not processed for user `05b2edb6fcb7496bb55650080a9da172`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:46.348474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-54ea6a36-69e7-4b8f-ad9f-0e61be41b22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:46.363375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1270/2538] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 13:08:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:08:46.375038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a3656-b32b-4393-9c36-bf1bdf4a8496 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:46.388688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a3656-b32b-4393-9c36-bf1bdf4a8496 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:46.392315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03ec57-b4ba-44b2-91d9-8ea04fcf7fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:46.392902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ec57-b4ba-44b2-91d9-8ea04fcf7fef service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:46.393183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ec57-b4ba-44b2-91d9-8ea04fcf7fef service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:46.393455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ec57-b4ba-44b2-91d9-8ea04fcf7fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:46.404512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03ec57-b4ba-44b2-91d9-8ea04fcf7fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:46.405945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03ec57-b4ba-44b2-91d9-8ea04fcf7fef service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:46.417389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1269/2539] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:08:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:08:48.107036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a3e34-35db-496b-81f1-822eef8e24f6 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:48.107824 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a3e34-35db-496b-81f1-822eef8e24f6 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:48.108113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a3e34-35db-496b-81f1-822eef8e24f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:48.128986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-4a6a3e34-35db-496b-81f1-822eef8e24f6 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:08:48.130344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4a6a3e34-35db-496b-81f1-822eef8e24f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:48.191317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1271/2540] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 13:08:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:08:48.200345 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03ec57-b4ba-44b2-91d9-8ea04fcf7fef service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:08:48.201680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ed03ec57-b4ba-44b2-91d9-8ea04fcf7fef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:08:48.274568 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03ec57-b4ba-44b2-91d9-8ea04fcf7fef service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:08:48.279766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2a720-128b-4c2d-9409-069991e8ff54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:08:48.280425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2a720-128b-4c2d-9409-069991e8ff54 service swift] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:08:48.280713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2a720-128b-4c2d-9409-069991e8ff54 service swift] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:08:48.280993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2a720-128b-4c2d-9409-069991e8ff54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:08:48.289328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2a720-128b-4c2d-9409-069991e8ff54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:08:48.290948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2a720-128b-4c2d-9409-069991e8ff54 service swift] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:08:48.299896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1270/2541] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:08:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:08:58.180946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=65 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:08:58.296308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=87 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:09:02.697848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6582f-65ec-4bdb-80d3-d0ef8b7ac986 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:02.698375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6582f-65ec-4bdb-80d3-d0ef8b7ac986 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:02.698375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6582f-65ec-4bdb-80d3-d0ef8b7ac986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:02.715568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-6fc6582f-65ec-4bdb-80d3-d0ef8b7ac986 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:02.716747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6fc6582f-65ec-4bdb-80d3-d0ef8b7ac986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:02.724054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1272/2542] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 13:09:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:02.735555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2a720-128b-4c2d-9409-069991e8ff54 service swift] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:02.746816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2a720-128b-4c2d-9409-069991e8ff54 service swift] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:02.751845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a285e5-b092-41e4-be0f-7d21c0d9b10f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:02.752483 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a285e5-b092-41e4-be0f-7d21c0d9b10f service swift] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:02.752769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a285e5-b092-41e4-be0f-7d21c0d9b10f service swift] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:02.753046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a285e5-b092-41e4-be0f-7d21c0d9b10f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:02.761762 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a285e5-b092-41e4-be0f-7d21c0d9b10f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:02.763355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a285e5-b092-41e4-be0f-7d21c0d9b10f service swift] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:02.771517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1271/2543] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:09:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:09:03.255264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a3570-156f-4251-abfb-bd13e923b2a4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:03.255555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a3570-156f-4251-abfb-bd13e923b2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:03.255838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a3570-156f-4251-abfb-bd13e923b2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:03.273632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-912a3570-156f-4251-abfb-bd13e923b2a4 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:03.275083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-912a3570-156f-4251-abfb-bd13e923b2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:03.282040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1273/2544] 10.222.0.45 () {58 vars in 967 bytes} [Tue May 24 13:09:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:03.294274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a285e5-b092-41e4-be0f-7d21c0d9b10f service swift] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:03.305559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a285e5-b092-41e4-be0f-7d21c0d9b10f service swift] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:03.310517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f31fe-5f2b-4b81-a4cb-af814f8343c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:03.311152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f31fe-5f2b-4b81-a4cb-af814f8343c7 service swift] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:03.311436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f31fe-5f2b-4b81-a4cb-af814f8343c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:03.311713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f31fe-5f2b-4b81-a4cb-af814f8343c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:03.319704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f31fe-5f2b-4b81-a4cb-af814f8343c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:03.321265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f31fe-5f2b-4b81-a4cb-af814f8343c7 service swift] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:03.329142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1272/2545] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:09:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:09:03.518097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d812fa-f030-4753-beaa-f5d1cfa7dafd None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:03.518619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d812fa-f030-4753-beaa-f5d1cfa7dafd None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:03.518898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d812fa-f030-4753-beaa-f5d1cfa7dafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:03.537949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-28d812fa-f030-4753-beaa-f5d1cfa7dafd None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:03.539183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-28d812fa-f030-4753-beaa-f5d1cfa7dafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:03.546625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1274/2546] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 13:09:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:03.558322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f31fe-5f2b-4b81-a4cb-af814f8343c7 service swift] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:03.569061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f31fe-5f2b-4b81-a4cb-af814f8343c7 service swift] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:03.574182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98234ec3-4840-49c2-acb5-74528de60181 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:03.574863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98234ec3-4840-49c2-acb5-74528de60181 service swift] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:03.575155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98234ec3-4840-49c2-acb5-74528de60181 service swift] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:03.575436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98234ec3-4840-49c2-acb5-74528de60181 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:03.584881 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98234ec3-4840-49c2-acb5-74528de60181 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:03.586483 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98234ec3-4840-49c2-acb5-74528de60181 service swift] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:03.594784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1273/2547] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:09:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:09:03.908486 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d23956-8e7d-4b83-a42d-9bfa18e07f15 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:03.908969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d23956-8e7d-4b83-a42d-9bfa18e07f15 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:03.909063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d23956-8e7d-4b83-a42d-9bfa18e07f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:03.926412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f2d23956-8e7d-4b83-a42d-9bfa18e07f15 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:03.927580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f2d23956-8e7d-4b83-a42d-9bfa18e07f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:03.934549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1275/2548] 10.222.0.45 () {58 vars in 967 bytes} [Tue May 24 13:09:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:03.946987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98234ec3-4840-49c2-acb5-74528de60181 service swift] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:03.957299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98234ec3-4840-49c2-acb5-74528de60181 service swift] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:03.962920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a113678-0aae-4208-8114-875337f478fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:03.963554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a113678-0aae-4208-8114-875337f478fb service swift] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:03.963838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a113678-0aae-4208-8114-875337f478fb service swift] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:03.964116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a113678-0aae-4208-8114-875337f478fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:03.972018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a113678-0aae-4208-8114-875337f478fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:03.973581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a113678-0aae-4208-8114-875337f478fb service swift] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:03.981611 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1274/2549] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:09:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:09:06.069509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d23956-8e7d-4b83-a42d-9bfa18e07f15 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:06.131056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d23956-8e7d-4b83-a42d-9bfa18e07f15 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:06.135623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:06.136500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:06.136788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:06.137069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] PATH_INFO: `/v3/users/b70413462fee46769f9c3f20b1d6881d` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:06.138275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b70413462fee46769f9c3f20b1d6881d)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:06.140382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:06.189164 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b70413462fee46769f9c3f20b1d6881d'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:09:06.196232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b70413462fee46769f9c3f20b1d6881d'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:09:06.196851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b70413462fee46769f9c3f20b1d6881d'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:09:06.197460 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b70413462fee46769f9c3f20b1d6881d'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:09:06.209246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b70413462fee46769f9c3f20b1d6881d'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:09:06.221167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1276/2550] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:09:06 2022] DELETE /identity/v3/users/b70413462fee46769f9c3f20b1d6881d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:09:06.226303 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a178fc-303e-431f-8aca-9c871b119e7f None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:06.226785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a178fc-303e-431f-8aca-9c871b119e7f None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:06.226882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a178fc-303e-431f-8aca-9c871b119e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:06.244964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-b7a178fc-303e-431f-8aca-9c871b119e7f None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:06.246169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b7a178fc-303e-431f-8aca-9c871b119e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:06.305313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1275/2551] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:06.314897 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:06.316276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:06.373761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132a3d8-d15a-4e27-8675-bd676ac0fa71 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:06.377616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c5a65-d889-442b-842d-db29008dd4b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:06.378392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c5a65-d889-442b-842d-db29008dd4b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:06.378710 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c5a65-d889-442b-842d-db29008dd4b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:06.378986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c5a65-d889-442b-842d-db29008dd4b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:06.386391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:09:06.389758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c5a65-d889-442b-842d-db29008dd4b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:06.391391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c5a65-d889-442b-842d-db29008dd4b0 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:06.403737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1277/2552] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:09:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:09:06.588261 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a178fc-303e-431f-8aca-9c871b119e7f None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:06.589700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b7a178fc-303e-431f-8aca-9c871b119e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:06.601979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a178fc-303e-431f-8aca-9c871b119e7f None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:06.607397 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:06.608844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:06.609286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:06.609571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] PATH_INFO: `/v3/projects/51cfc9ec9dcc49c0b16229698640b951` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:06.611348 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=51cfc9ec9dcc49c0b16229698640b951)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:06.613581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:06.675365 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51cfc9ec9dcc49c0b16229698640b951', 'request_id': 'req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:09:06.682804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51cfc9ec9dcc49c0b16229698640b951', 'request_id': 'req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:09:06.690935 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51cfc9ec9dcc49c0b16229698640b951', 'request_id': 'req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:09:06.703971 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] The token cache is being invalidate because project 51cfc9ec9dcc49c0b16229698640b951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:09:06.704259 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:09:06.704925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6574438a-4f5f-4797-a0d4-9ad16ebd5c0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:09:06.706996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1276/2553] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:09:06 2022] DELETE /identity/v3/projects/51cfc9ec9dcc49c0b16229698640b951 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:09:06.712143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ac3a9-99f0-43d8-9148-59dc923207e4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:06.712537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ac3a9-99f0-43d8-9148-59dc923207e4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:06.712822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ac3a9-99f0-43d8-9148-59dc923207e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:06.731897 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-270ac3a9-99f0-43d8-9148-59dc923207e4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:06.733108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-270ac3a9-99f0-43d8-9148-59dc923207e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:06.796641 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1278/2554] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:06.804337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43396aa9-c72a-42d9-88ba-fcb4d4feca84 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:06.804634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43396aa9-c72a-42d9-88ba-fcb4d4feca84 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:06.805023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43396aa9-c72a-42d9-88ba-fcb4d4feca84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:06.823783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-43396aa9-c72a-42d9-88ba-fcb4d4feca84 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:06.824908 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-43396aa9-c72a-42d9-88ba-fcb4d4feca84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:06.833728 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1277/2555] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:06.838182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ac3a9-99f0-43d8-9148-59dc923207e4 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:06.847850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ac3a9-99f0-43d8-9148-59dc923207e4 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:06.852677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44c728-96cb-4b8f-87f7-c2d776ed76fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:06.853336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44c728-96cb-4b8f-87f7-c2d776ed76fe admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:06.853626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44c728-96cb-4b8f-87f7-c2d776ed76fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:06.853905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44c728-96cb-4b8f-87f7-c2d776ed76fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:06.854665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44c728-96cb-4b8f-87f7-c2d776ed76fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:09:06.854978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44c728-96cb-4b8f-87f7-c2d776ed76fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:06.856580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44c728-96cb-4b8f-87f7-c2d776ed76fe admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:06.868232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1279/2556] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:09:06.872386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43396aa9-c72a-42d9-88ba-fcb4d4feca84 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:06.881910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43396aa9-c72a-42d9-88ba-fcb4d4feca84 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:06.886531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4cf15-98f6-4ea8-9c47-3d573753cac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:06.887209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4cf15-98f6-4ea8-9c47-3d573753cac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:06.887496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4cf15-98f6-4ea8-9c47-3d573753cac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:06.887771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4cf15-98f6-4ea8-9c47-3d573753cac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:06.888458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4cf15-98f6-4ea8-9c47-3d573753cac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:06.890382 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4cf15-98f6-4ea8-9c47-3d573753cac9 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:06.915467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1278/2557] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:09:06 2022] POST /identity/v3/projects => generated 385 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:09:06.919493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44c728-96cb-4b8f-87f7-c2d776ed76fe admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:06.930939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44c728-96cb-4b8f-87f7-c2d776ed76fe admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:06.991412 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2414e8-5b8f-4d7e-a0c8-007e2620dc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:06.992147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2414e8-5b8f-4d7e-a0c8-007e2620dc26 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:06.992434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2414e8-5b8f-4d7e-a0c8-007e2620dc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:06.992711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2414e8-5b8f-4d7e-a0c8-007e2620dc26 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:06.993392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2414e8-5b8f-4d7e-a0c8-007e2620dc26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:06.995484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2414e8-5b8f-4d7e-a0c8-007e2620dc26 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:07.019971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1280/2558] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:09:06 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:09:07.021113 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4cf15-98f6-4ea8-9c47-3d573753cac9 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:07.026789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:09:07.033007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4cf15-98f6-4ea8-9c47-3d573753cac9 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:07.039077 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015c011-c7ec-46bf-a1a3-203f2cc59440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:07.039820 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015c011-c7ec-46bf-a1a3-203f2cc59440 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:07.040154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015c011-c7ec-46bf-a1a3-203f2cc59440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:07.040436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015c011-c7ec-46bf-a1a3-203f2cc59440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:07.041225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015c011-c7ec-46bf-a1a3-203f2cc59440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:07.043237 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015c011-c7ec-46bf-a1a3-203f2cc59440 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:07.057589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1279/2559] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:09:07.058836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2414e8-5b8f-4d7e-a0c8-007e2620dc26 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:07.069786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2414e8-5b8f-4d7e-a0c8-007e2620dc26 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:07.074724 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d70f6-6bf5-4558-9047-84852a1bc9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:07.075332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d70f6-6bf5-4558-9047-84852a1bc9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:07.075618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d70f6-6bf5-4558-9047-84852a1bc9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:07.075894 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d70f6-6bf5-4558-9047-84852a1bc9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:07.076651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d70f6-6bf5-4558-9047-84852a1bc9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:07.078326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d70f6-6bf5-4558-9047-84852a1bc9ef admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:07.089847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1281/2560] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:09:07.093985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015c011-c7ec-46bf-a1a3-203f2cc59440 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:07.104040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015c011-c7ec-46bf-a1a3-203f2cc59440 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:07.108805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e7f21-2840-4990-a671-d53b6b8da124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:07.109769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e7f21-2840-4990-a671-d53b6b8da124 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:07.110117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e7f21-2840-4990-a671-d53b6b8da124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:07.110495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e7f21-2840-4990-a671-d53b6b8da124 admin admin] PATH_INFO: `/v3/projects/0df8852ede3b4845b824cc9345afbb5d/users/66b0a0ef7ea24caaa9540786a4d28682/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:07.126243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e7f21-2840-4990-a671-d53b6b8da124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df8852ede3b4845b824cc9345afbb5d, user_id=66b0a0ef7ea24caaa9540786a4d28682, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:07.129185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e7f21-2840-4990-a671-d53b6b8da124 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:07.143281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1280/2561] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:09:07 2022] PUT /identity/v3/projects/0df8852ede3b4845b824cc9345afbb5d/users/66b0a0ef7ea24caaa9540786a4d28682/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:09:07.148356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f27e8-0253-4dd4-ac55-2c6c5a835c94 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:07.148653 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f27e8-0253-4dd4-ac55-2c6c5a835c94 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:07.148934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f27e8-0253-4dd4-ac55-2c6c5a835c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:07.169988 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a47f27e8-0253-4dd4-ac55-2c6c5a835c94 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:07.171378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a47f27e8-0253-4dd4-ac55-2c6c5a835c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:07.235458 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1282/2562] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:07.245738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e7f21-2840-4990-a671-d53b6b8da124 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:07.247462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-038e7f21-2840-4990-a671-d53b6b8da124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:07.308741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e7f21-2840-4990-a671-d53b6b8da124 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:07.312464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e156112-6cf8-4246-8079-138254a85b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:07.313117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e156112-6cf8-4246-8079-138254a85b4e service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:07.313406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e156112-6cf8-4246-8079-138254a85b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:07.313688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e156112-6cf8-4246-8079-138254a85b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:07.323904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e156112-6cf8-4246-8079-138254a85b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:07.325539 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e156112-6cf8-4246-8079-138254a85b4e service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:07.335716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1281/2563] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:09:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:09:11.571670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4512196-f780-408f-a39e-6207a51204d9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:11.572513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4512196-f780-408f-a39e-6207a51204d9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:11.572901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4512196-f780-408f-a39e-6207a51204d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:11.579559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:09:11.594060 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b4512196-f780-408f-a39e-6207a51204d9 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:11.595656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b4512196-f780-408f-a39e-6207a51204d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:11.666322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1283/2564] 10.222.0.45 () {58 vars in 967 bytes} [Tue May 24 13:09:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:11.679944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e156112-6cf8-4246-8079-138254a85b4e service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:11.682012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5e156112-6cf8-4246-8079-138254a85b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:11.687316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3518-e5c5-4df6-a5d1-f1ea5b65d1f8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:11.687606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3518-e5c5-4df6-a5d1-f1ea5b65d1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:11.687881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3518-e5c5-4df6-a5d1-f1ea5b65d1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:11.716573 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-bdfc3518-e5c5-4df6-a5d1-f1ea5b65d1f8 None None] MFA Rules not processed for user `66b0a0ef7ea24caaa9540786a4d28682`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:11.717784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bdfc3518-e5c5-4df6-a5d1-f1ea5b65d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:11.769680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e156112-6cf8-4246-8079-138254a85b4e service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:11.775204 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d3c22-87c3-4336-b598-76c87e9af9d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:11.776028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d3c22-87c3-4336-b598-76c87e9af9d1 service swift] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:11.776323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d3c22-87c3-4336-b598-76c87e9af9d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:11.776606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d3c22-87c3-4336-b598-76c87e9af9d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:11.786736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d3c22-87c3-4336-b598-76c87e9af9d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:11.788462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d3c22-87c3-4336-b598-76c87e9af9d1 service swift] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:11.806964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1282/2565] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:09:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:09:11.828792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1284/2566] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:09:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:09:11.833470 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d3c22-87c3-4336-b598-76c87e9af9d1 service swift] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:11.844719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d3c22-87c3-4336-b598-76c87e9af9d1 service swift] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:11.850271 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb304ae6-2cb0-4338-a1cd-abd3505e2fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:11.850878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb304ae6-2cb0-4338-a1cd-abd3505e2fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:11.851164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb304ae6-2cb0-4338-a1cd-abd3505e2fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:11.851443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb304ae6-2cb0-4338-a1cd-abd3505e2fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:11.852157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb304ae6-2cb0-4338-a1cd-abd3505e2fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:11.854803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb304ae6-2cb0-4338-a1cd-abd3505e2fa3 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:11.896549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3518-e5c5-4df6-a5d1-f1ea5b65d1f8 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:11.896672 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1283/2567] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:09:11 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:09:11.908661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3518-e5c5-4df6-a5d1-f1ea5b65d1f8 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:11.994992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7081b8a-5755-4eb2-bdd2-c1deb2a065ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:11.995810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7081b8a-5755-4eb2-bdd2-c1deb2a065ac admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:11.996112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7081b8a-5755-4eb2-bdd2-c1deb2a065ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:11.996394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7081b8a-5755-4eb2-bdd2-c1deb2a065ac admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:11.997113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7081b8a-5755-4eb2-bdd2-c1deb2a065ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:11.999281 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7081b8a-5755-4eb2-bdd2-c1deb2a065ac admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:12.026325 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1285/2568] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:09:11 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:09:12.031913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb304ae6-2cb0-4338-a1cd-abd3505e2fa3 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:12.050495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb304ae6-2cb0-4338-a1cd-abd3505e2fa3 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:12.059417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c89a3-bcd4-454d-9367-de426eb7302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:12.060103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c89a3-bcd4-454d-9367-de426eb7302d admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:12.060398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c89a3-bcd4-454d-9367-de426eb7302d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:12.060678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c89a3-bcd4-454d-9367-de426eb7302d admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:12.061417 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c89a3-bcd4-454d-9367-de426eb7302d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:12.069925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c89a3-bcd4-454d-9367-de426eb7302d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:12.095723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1284/2569] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:09:12.100630 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7081b8a-5755-4eb2-bdd2-c1deb2a065ac admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:12.116371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7081b8a-5755-4eb2-bdd2-c1deb2a065ac admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:12.124422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a30462-aa37-4535-8060-dbfbdde73c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:12.125333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30462-aa37-4535-8060-dbfbdde73c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:12.126323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30462-aa37-4535-8060-dbfbdde73c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:12.127077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30462-aa37-4535-8060-dbfbdde73c88 admin admin] PATH_INFO: `/v3/projects/a067c6767fc848b39d7ab93777cdc1b3/users/d165c1a66ad245fcb611d96a290b6162/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:12.151632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a30462-aa37-4535-8060-dbfbdde73c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a067c6767fc848b39d7ab93777cdc1b3, user_id=d165c1a66ad245fcb611d96a290b6162, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:12.155461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a30462-aa37-4535-8060-dbfbdde73c88 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:12.174054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1286/2570] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:09:12 2022] PUT /identity/v3/projects/a067c6767fc848b39d7ab93777cdc1b3/users/d165c1a66ad245fcb611d96a290b6162/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:09:12.179313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c89a3-bcd4-454d-9367-de426eb7302d admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:12.191046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c89a3-bcd4-454d-9367-de426eb7302d admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:12.255510 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:09:12.343562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730e414-88b8-495d-9119-5b595f296fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:12.344440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730e414-88b8-495d-9119-5b595f296fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:12.344799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730e414-88b8-495d-9119-5b595f296fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:12.345176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730e414-88b8-495d-9119-5b595f296fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:12.346002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730e414-88b8-495d-9119-5b595f296fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:12.347993 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730e414-88b8-495d-9119-5b595f296fcb admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:12.365411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1285/2571] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:09:12.366677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a30462-aa37-4535-8060-dbfbdde73c88 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:12.374468 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:09:12.378503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a30462-aa37-4535-8060-dbfbdde73c88 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:12.384624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ca40b-07d3-4c43-8c60-5fc81b9b6a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:12.385523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ca40b-07d3-4c43-8c60-5fc81b9b6a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:12.385805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ca40b-07d3-4c43-8c60-5fc81b9b6a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:12.386087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ca40b-07d3-4c43-8c60-5fc81b9b6a7e admin admin] PATH_INFO: `/v3/projects/a067c6767fc848b39d7ab93777cdc1b3/users/d165c1a66ad245fcb611d96a290b6162/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:12.388151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ca40b-07d3-4c43-8c60-5fc81b9b6a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a067c6767fc848b39d7ab93777cdc1b3, user_id=d165c1a66ad245fcb611d96a290b6162, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:12.391027 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ca40b-07d3-4c43-8c60-5fc81b9b6a7e admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:12.402572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1287/2572] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:09:12 2022] PUT /identity/v3/projects/a067c6767fc848b39d7ab93777cdc1b3/users/d165c1a66ad245fcb611d96a290b6162/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:09:17.780009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61259b-a82a-43eb-8a05-fb92b0be3eb4 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:17.780538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61259b-a82a-43eb-8a05-fb92b0be3eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:17.780626 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61259b-a82a-43eb-8a05-fb92b0be3eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:17.813770 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-ae61259b-a82a-43eb-8a05-fb92b0be3eb4 None None] MFA Rules not processed for user `d165c1a66ad245fcb611d96a290b6162`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:17.815437 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ae61259b-a82a-43eb-8a05-fb92b0be3eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:17.907900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1286/2573] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:09:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:17.920924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ca40b-07d3-4c43-8c60-5fc81b9b6a7e admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:17.922746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b90ca40b-07d3-4c43-8c60-5fc81b9b6a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:17.985785 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ca40b-07d3-4c43-8c60-5fc81b9b6a7e admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:17.989959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46672a-bc89-48ff-8b36-9b7beb012419 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e47f1d65687e4cfcbce764076141702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:17.990682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46672a-bc89-48ff-8b36-9b7beb012419 service glance] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:17.990968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46672a-bc89-48ff-8b36-9b7beb012419 service glance] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:17.991245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46672a-bc89-48ff-8b36-9b7beb012419 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:18.061664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46672a-bc89-48ff-8b36-9b7beb012419 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:18.063299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46672a-bc89-48ff-8b36-9b7beb012419 service glance] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:18.073348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1288/2574] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:09:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:09:18.186354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f748c-3945-4389-a3ae-7a41b6d93b53 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:18.187154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f748c-3945-4389-a3ae-7a41b6d93b53 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:18.187523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f748c-3945-4389-a3ae-7a41b6d93b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:18.208737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-8d0f748c-3945-4389-a3ae-7a41b6d93b53 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:18.210141 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8d0f748c-3945-4389-a3ae-7a41b6d93b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:18.276892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1287/2575] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 13:09:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:18.291216 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46672a-bc89-48ff-8b36-9b7beb012419 service glance] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:18.365886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46672a-bc89-48ff-8b36-9b7beb012419 service glance] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:18.370892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de578e-20c6-41b9-88af-99310937f5cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:18.371537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de578e-20c6-41b9-88af-99310937f5cd service swift] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:18.371822 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de578e-20c6-41b9-88af-99310937f5cd service swift] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:18.372096 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de578e-20c6-41b9-88af-99310937f5cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:18.381042 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de578e-20c6-41b9-88af-99310937f5cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:18.382683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de578e-20c6-41b9-88af-99310937f5cd service swift] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:18.391584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1289/2576] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:09:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:09:21.562591 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f748c-3945-4389-a3ae-7a41b6d93b53 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:21.565171 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8d0f748c-3945-4389-a3ae-7a41b6d93b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:21.644246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f748c-3945-4389-a3ae-7a41b6d93b53 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:21.650620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3be789-0449-42f5-8df5-888dfd2be89a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:21.651751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3be789-0449-42f5-8df5-888dfd2be89a service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:21.652242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3be789-0449-42f5-8df5-888dfd2be89a service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:21.652648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3be789-0449-42f5-8df5-888dfd2be89a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:21.655011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-da3be789-0449-42f5-8df5-888dfd2be89a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:21.661414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:09:21.721851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3be789-0449-42f5-8df5-888dfd2be89a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:21.723649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3be789-0449-42f5-8df5-888dfd2be89a service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:21.734060 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1288/2577] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:09:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:09:27.096066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dde37-3beb-4cf9-86df-ad88b61232ec None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:27.096558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dde37-3beb-4cf9-86df-ad88b61232ec None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:27.096658 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dde37-3beb-4cf9-86df-ad88b61232ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:27.114353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ed9dde37-3beb-4cf9-86df-ad88b61232ec None None] MFA Rules not processed for user `66b0a0ef7ea24caaa9540786a4d28682`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:09:27.115556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ed9dde37-3beb-4cf9-86df-ad88b61232ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:27.122356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1290/2578] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:09:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:09:27.378824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3be789-0449-42f5-8df5-888dfd2be89a service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:27.380265 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-da3be789-0449-42f5-8df5-888dfd2be89a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:27.438150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3be789-0449-42f5-8df5-888dfd2be89a service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:27.442357 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f32f1f-9125-44f3-aaf7-0d7d4060763d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:27.443002 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f32f1f-9125-44f3-aaf7-0d7d4060763d service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:27.443284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f32f1f-9125-44f3-aaf7-0d7d4060763d service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:27.443558 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f32f1f-9125-44f3-aaf7-0d7d4060763d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:27.445329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-85f32f1f-9125-44f3-aaf7-0d7d4060763d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:27.455053 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f32f1f-9125-44f3-aaf7-0d7d4060763d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:27.456404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f32f1f-9125-44f3-aaf7-0d7d4060763d service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:27.464215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1289/2579] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:09:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:09:37.108675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:09:37.460784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=50 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:09:57.321735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dde37-3beb-4cf9-86df-ad88b61232ec None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:09:57.331752 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dde37-3beb-4cf9-86df-ad88b61232ec None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:09:57.335937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f74f4-ac1e-4232-9889-b4c920bbd4a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:09:57.336577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f74f4-ac1e-4232-9889-b4c920bbd4a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:09:57.336861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f74f4-ac1e-4232-9889-b4c920bbd4a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:09:57.337137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f74f4-ac1e-4232-9889-b4c920bbd4a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:09:57.338893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e41f74f4-ac1e-4232-9889-b4c920bbd4a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:09:57.418532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f74f4-ac1e-4232-9889-b4c920bbd4a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:09:57.419996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f74f4-ac1e-4232-9889-b4c920bbd4a6 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:09:57.431046 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1291/2580] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:09:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:10:00.735678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f32f1f-9125-44f3-aaf7-0d7d4060763d service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:10:00.745126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f32f1f-9125-44f3-aaf7-0d7d4060763d service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:10:00.748773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f97b5-0be6-42b1-b3a8-beb70f1f0e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:10:00.749404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f97b5-0be6-42b1-b3a8-beb70f1f0e94 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:10:00.749685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f97b5-0be6-42b1-b3a8-beb70f1f0e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:10:00.749960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f97b5-0be6-42b1-b3a8-beb70f1f0e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:10:00.751750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-810f97b5-0be6-42b1-b3a8-beb70f1f0e94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:10:00.765356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f97b5-0be6-42b1-b3a8-beb70f1f0e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:10:00.766789 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f97b5-0be6-42b1-b3a8-beb70f1f0e94 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:10:00.777407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1290/2581] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:10:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:10:07.426036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=34 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:10:10.755621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f74f4-ac1e-4232-9889-b4c920bbd4a6 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:10:10.757063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e41f74f4-ac1e-4232-9889-b4c920bbd4a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:10:10.768449 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f74f4-ac1e-4232-9889-b4c920bbd4a6 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:10:10.772139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e41961-7ce6-4557-afe4-2e8b2533f957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:10:10.772645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:10:10.773045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e41961-7ce6-4557-afe4-2e8b2533f957 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:10:10.773150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e41961-7ce6-4557-afe4-2e8b2533f957 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:10:10.773347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e41961-7ce6-4557-afe4-2e8b2533f957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:10:10.775176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-47e41961-7ce6-4557-afe4-2e8b2533f957 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:10:10.854894 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e41961-7ce6-4557-afe4-2e8b2533f957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:10:10.856250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e41961-7ce6-4557-afe4-2e8b2533f957 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:10:10.874856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1292/2582] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:10:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:10:14.309666 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333c646-e340-45b8-af60-eabb589dd003 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:10:14.309666 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333c646-e340-45b8-af60-eabb589dd003 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:10:14.309666 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333c646-e340-45b8-af60-eabb589dd003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:10:14.358233 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-b333c646-e340-45b8-af60-eabb589dd003 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:10:14.361286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b333c646-e340-45b8-af60-eabb589dd003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:10:14.410435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1291/2583] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 13:10:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:10:14.415321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e41961-7ce6-4557-afe4-2e8b2533f957 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:10:14.431266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e41961-7ce6-4557-afe4-2e8b2533f957 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:10:14.439027 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210d4db-acb8-437b-a730-41a2547ed333 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:10:14.440087 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210d4db-acb8-437b-a730-41a2547ed333 service swift] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:10:14.442513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210d4db-acb8-437b-a730-41a2547ed333 service swift] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:10:14.444193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210d4db-acb8-437b-a730-41a2547ed333 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:10:14.456117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210d4db-acb8-437b-a730-41a2547ed333 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:10:14.459204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210d4db-acb8-437b-a730-41a2547ed333 service swift] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:10:14.503780 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1293/2584] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:10:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:10:24.339824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:10:24.466342 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=47 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:10:29.092248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333c646-e340-45b8-af60-eabb589dd003 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:10:29.102099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333c646-e340-45b8-af60-eabb589dd003 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:10:29.105840 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a83bc-96be-4d77-8348-a26bf1dbd8ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:10:29.106612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a83bc-96be-4d77-8348-a26bf1dbd8ad service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:10:29.106895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a83bc-96be-4d77-8348-a26bf1dbd8ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:10:29.107169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a83bc-96be-4d77-8348-a26bf1dbd8ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:10:29.108995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-187a83bc-96be-4d77-8348-a26bf1dbd8ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:10:29.120041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a83bc-96be-4d77-8348-a26bf1dbd8ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:10:29.121843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a83bc-96be-4d77-8348-a26bf1dbd8ad service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:10:29.130773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1292/2585] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:10:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:10:39.126563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:10:59.805229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210d4db-acb8-437b-a730-41a2547ed333 service swift] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:10:59.815098 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210d4db-acb8-437b-a730-41a2547ed333 service swift] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:10:59.818616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a73649-ee52-4faa-9dc1-4040dd3ae620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:10:59.819245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a73649-ee52-4faa-9dc1-4040dd3ae620 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:10:59.819523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a73649-ee52-4faa-9dc1-4040dd3ae620 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:10:59.819794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a73649-ee52-4faa-9dc1-4040dd3ae620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:10:59.821545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b4a73649-ee52-4faa-9dc1-4040dd3ae620 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:10:59.901793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a73649-ee52-4faa-9dc1-4040dd3ae620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:10:59.903258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a73649-ee52-4faa-9dc1-4040dd3ae620 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:10:59.922009 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1294/2586] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:10:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:11:09.919257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=41 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:13:13.199014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a83bc-96be-4d77-8348-a26bf1dbd8ad service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:13:13.200389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-187a83bc-96be-4d77-8348-a26bf1dbd8ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:13:13.340006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a83bc-96be-4d77-8348-a26bf1dbd8ad service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:13:13.345038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f181ec-d8bc-4a0a-a697-c38cc125703e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:13:13.345676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f181ec-d8bc-4a0a-a697-c38cc125703e service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:13:13.345957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f181ec-d8bc-4a0a-a697-c38cc125703e service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:13:13.346229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f181ec-d8bc-4a0a-a697-c38cc125703e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:13:13.347994 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-29f181ec-d8bc-4a0a-a697-c38cc125703e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:13:13.359055 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f181ec-d8bc-4a0a-a697-c38cc125703e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:13:13.361641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f181ec-d8bc-4a0a-a697-c38cc125703e service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:13:13.371576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1293/2587] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:13:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:13:23.367155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=63 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:14:17.056522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a73649-ee52-4faa-9dc1-4040dd3ae620 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:14:17.085032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a73649-ee52-4faa-9dc1-4040dd3ae620 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:14:17.089286 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b4a65-b8a8-413f-b92a-ba5e4e580c83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:14:17.089935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b4a65-b8a8-413f-b92a-ba5e4e580c83 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:14:17.090225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b4a65-b8a8-413f-b92a-ba5e4e580c83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:14:17.090531 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b4a65-b8a8-413f-b92a-ba5e4e580c83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:14:17.099694 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b4a65-b8a8-413f-b92a-ba5e4e580c83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:14:17.101143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b4a65-b8a8-413f-b92a-ba5e4e580c83 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:14:17.110317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1295/2588] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:14:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:14:21.219290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f181ec-d8bc-4a0a-a697-c38cc125703e service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:14:21.229421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f181ec-d8bc-4a0a-a697-c38cc125703e service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:14:21.233001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49109cf2-f4cf-459d-b5c9-8bfa161fc54c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:14:21.233641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49109cf2-f4cf-459d-b5c9-8bfa161fc54c service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:14:21.233925 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49109cf2-f4cf-459d-b5c9-8bfa161fc54c service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:14:21.234203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49109cf2-f4cf-459d-b5c9-8bfa161fc54c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:14:21.242780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49109cf2-f4cf-459d-b5c9-8bfa161fc54c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:14:21.244231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49109cf2-f4cf-459d-b5c9-8bfa161fc54c service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:14:21.279390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1294/2589] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:14:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:14:27.106043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=15 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:14:31.276952 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=13 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:15:10.723605 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b4a65-b8a8-413f-b92a-ba5e4e580c83 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:15:10.732747 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b4a65-b8a8-413f-b92a-ba5e4e580c83 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:15:10.736309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca202a-a07c-4439-9aea-0698dd6c28f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:15:10.736945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca202a-a07c-4439-9aea-0698dd6c28f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:15:10.737225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca202a-a07c-4439-9aea-0698dd6c28f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:15:10.737499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca202a-a07c-4439-9aea-0698dd6c28f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:15:10.746025 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca202a-a07c-4439-9aea-0698dd6c28f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:15:10.747495 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca202a-a07c-4439-9aea-0698dd6c28f5 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:15:10.756387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1296/2590] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:15:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:15:20.752205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=6 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:16:00.135085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49109cf2-f4cf-459d-b5c9-8bfa161fc54c service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:16:00.143861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49109cf2-f4cf-459d-b5c9-8bfa161fc54c service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:16:00.147506 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ce5ec-9684-4cbb-8c11-d9acc2ad778c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:16:00.148140 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ce5ec-9684-4cbb-8c11-d9acc2ad778c service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:16:00.148423 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ce5ec-9684-4cbb-8c11-d9acc2ad778c service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:16:00.148694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ce5ec-9684-4cbb-8c11-d9acc2ad778c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:16:00.158532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ce5ec-9684-4cbb-8c11-d9acc2ad778c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:16:00.159960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ce5ec-9684-4cbb-8c11-d9acc2ad778c service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:16:00.170184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1295/2591] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:16:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:16:10.164978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:16:15.092277 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca202a-a07c-4439-9aea-0698dd6c28f5 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:16:15.100732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca202a-a07c-4439-9aea-0698dd6c28f5 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:16:15.104331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968a7b9-5464-4adf-947a-c7b7a65de2bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:16:15.104969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968a7b9-5464-4adf-947a-c7b7a65de2bc service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:16:15.105248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968a7b9-5464-4adf-947a-c7b7a65de2bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:16:15.105521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968a7b9-5464-4adf-947a-c7b7a65de2bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:16:15.123507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968a7b9-5464-4adf-947a-c7b7a65de2bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:16:15.124936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968a7b9-5464-4adf-947a-c7b7a65de2bc service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:16:15.145181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1297/2592] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:16:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:16:19.923863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ce5ec-9684-4cbb-8c11-d9acc2ad778c service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:16:19.943043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ce5ec-9684-4cbb-8c11-d9acc2ad778c service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:16:19.946630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae1e18-ae0f-4723-b8fa-28c5df6cc9fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:16:19.947269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae1e18-ae0f-4723-b8fa-28c5df6cc9fe service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:16:19.947556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae1e18-ae0f-4723-b8fa-28c5df6cc9fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:16:19.947834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae1e18-ae0f-4723-b8fa-28c5df6cc9fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:16:19.955804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae1e18-ae0f-4723-b8fa-28c5df6cc9fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:16:19.957235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae1e18-ae0f-4723-b8fa-28c5df6cc9fe service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:16:19.965115 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1296/2593] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:16:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:16:25.142700 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=13 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:16:29.961631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=10 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:18:21.216289 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968a7b9-5464-4adf-947a-c7b7a65de2bc service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:18:21.217659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4968a7b9-5464-4adf-947a-c7b7a65de2bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:18:21.231563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968a7b9-5464-4adf-947a-c7b7a65de2bc service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:18:21.236440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51652-903b-4d85-a264-95120268133c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:18:21.237069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51652-903b-4d85-a264-95120268133c service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:18:21.237347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51652-903b-4d85-a264-95120268133c service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:18:21.237618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51652-903b-4d85-a264-95120268133c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:18:21.246255 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51652-903b-4d85-a264-95120268133c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:18:21.247849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51652-903b-4d85-a264-95120268133c service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:18:21.257940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1298/2594] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:18:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:18:31.253344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=6 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:19:17.476421 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae1e18-ae0f-4723-b8fa-28c5df6cc9fe service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:19:17.485726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae1e18-ae0f-4723-b8fa-28c5df6cc9fe service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:19:17.489650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab8f83-d62d-4186-8ed6-8d873fdc64cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:19:17.490334 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab8f83-d62d-4186-8ed6-8d873fdc64cd service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:19:17.490619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab8f83-d62d-4186-8ed6-8d873fdc64cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:19:17.490891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab8f83-d62d-4186-8ed6-8d873fdc64cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:19:17.492719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1cab8f83-d62d-4186-8ed6-8d873fdc64cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:19:17.524687 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab8f83-d62d-4186-8ed6-8d873fdc64cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:19:17.526830 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab8f83-d62d-4186-8ed6-8d873fdc64cd service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:19:17.544493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1297/2595] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:19:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:19:21.813714 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51652-903b-4d85-a264-95120268133c service placement] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:19:21.867072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51652-903b-4d85-a264-95120268133c service placement] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:19:21.870868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56838e8a-cf13-45ed-9979-20b0f676157f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:19:21.871502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56838e8a-cf13-45ed-9979-20b0f676157f service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:19:21.871787 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56838e8a-cf13-45ed-9979-20b0f676157f service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:19:21.872061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56838e8a-cf13-45ed-9979-20b0f676157f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:19:21.873936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-56838e8a-cf13-45ed-9979-20b0f676157f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:19:21.944916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56838e8a-cf13-45ed-9979-20b0f676157f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:19:21.946388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56838e8a-cf13-45ed-9979-20b0f676157f service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:19:21.955763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1299/2596] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:19:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:19:27.542472 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=19 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:19:31.951653 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=53 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:20:11.044578 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab8f83-d62d-4186-8ed6-8d873fdc64cd service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:20:11.046140 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1cab8f83-d62d-4186-8ed6-8d873fdc64cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:20:11.112432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab8f83-d62d-4186-8ed6-8d873fdc64cd service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:20:11.116406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d59d65-30e2-407b-869b-7e4f2a24a6b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:20:11.117050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d59d65-30e2-407b-869b-7e4f2a24a6b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:20:11.117338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d59d65-30e2-407b-869b-7e4f2a24a6b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:20:11.117617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d59d65-30e2-407b-869b-7e4f2a24a6b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:20:11.119507 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f9d59d65-30e2-407b-869b-7e4f2a24a6b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:20:11.196647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d59d65-30e2-407b-869b-7e4f2a24a6b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:20:11.198000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d59d65-30e2-407b-869b-7e4f2a24a6b8 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:20:11.216990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1298/2597] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:20:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:20:21.213812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=66 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:21:00.285392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56838e8a-cf13-45ed-9979-20b0f676157f service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:21:00.286853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-56838e8a-cf13-45ed-9979-20b0f676157f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:21:00.375097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56838e8a-cf13-45ed-9979-20b0f676157f service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:21:00.380521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b34a0-2019-4ba5-97fd-bb80519aac73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:21:00.381102 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b34a0-2019-4ba5-97fd-bb80519aac73 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:21:00.381410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b34a0-2019-4ba5-97fd-bb80519aac73 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:21:00.381683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b34a0-2019-4ba5-97fd-bb80519aac73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:21:00.383544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b79b34a0-2019-4ba5-97fd-bb80519aac73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:21:00.461827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b34a0-2019-4ba5-97fd-bb80519aac73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:21:00.463483 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b34a0-2019-4ba5-97fd-bb80519aac73 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:21:00.481615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1300/2598] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:21:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:21:10.479006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=74 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:21:22.148356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d59d65-30e2-407b-869b-7e4f2a24a6b8 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:21:22.162190 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d59d65-30e2-407b-869b-7e4f2a24a6b8 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:21:22.168774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17d70d-4ec3-4721-9cdb-ae89ca6c3494 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:21:22.169440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17d70d-4ec3-4721-9cdb-ae89ca6c3494 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:21:22.169730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17d70d-4ec3-4721-9cdb-ae89ca6c3494 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:21:22.170010 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17d70d-4ec3-4721-9cdb-ae89ca6c3494 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:21:22.171865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ec17d70d-4ec3-4721-9cdb-ae89ca6c3494 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:21:22.184777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17d70d-4ec3-4721-9cdb-ae89ca6c3494 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:21:22.186346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17d70d-4ec3-4721-9cdb-ae89ca6c3494 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:21:22.197488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1299/2599] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:21:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:21:22.266513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b34a0-2019-4ba5-97fd-bb80519aac73 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:21:22.268137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b79b34a0-2019-4ba5-97fd-bb80519aac73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:21:22.329701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17d70d-4ec3-4721-9cdb-ae89ca6c3494 service nova] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:21:22.332812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ec17d70d-4ec3-4721-9cdb-ae89ca6c3494 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:21:22.352605 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b34a0-2019-4ba5-97fd-bb80519aac73 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:21:22.356757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d1b21-7abd-4bec-a52a-6686b9499492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:21:22.357422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d1b21-7abd-4bec-a52a-6686b9499492 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:21:22.357730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d1b21-7abd-4bec-a52a-6686b9499492 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:21:22.357969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d1b21-7abd-4bec-a52a-6686b9499492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:21:22.360040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-413d1b21-7abd-4bec-a52a-6686b9499492 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:21:22.379077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d1b21-7abd-4bec-a52a-6686b9499492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:21:22.380733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d1b21-7abd-4bec-a52a-6686b9499492 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:21:22.393958 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1301/2600] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:21:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:21:22.412363 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17d70d-4ec3-4721-9cdb-ae89ca6c3494 service nova] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:21:22.415010 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafe9a9-951f-48d3-8b90-70de70ce663d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:21:22.415732 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafe9a9-951f-48d3-8b90-70de70ce663d service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:21:22.416018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafe9a9-951f-48d3-8b90-70de70ce663d service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:21:22.416298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafe9a9-951f-48d3-8b90-70de70ce663d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:21:22.418262 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-dcafe9a9-951f-48d3-8b90-70de70ce663d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:21:22.432498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcafe9a9-951f-48d3-8b90-70de70ce663d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:21:22.433944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcafe9a9-951f-48d3-8b90-70de70ce663d service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:21:22.445610 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1300/2601] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:21:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:21:24.752070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d1b21-7abd-4bec-a52a-6686b9499492 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:21:24.761002 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d1b21-7abd-4bec-a52a-6686b9499492 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:21:24.764893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b60cf-de3e-464d-891e-aa0e5f397b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:21:24.765689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b60cf-de3e-464d-891e-aa0e5f397b1f service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:21:24.765970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b60cf-de3e-464d-891e-aa0e5f397b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:21:24.766275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b60cf-de3e-464d-891e-aa0e5f397b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:21:24.768714 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0d5b60cf-de3e-464d-891e-aa0e5f397b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:21:24.783577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b60cf-de3e-464d-891e-aa0e5f397b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:21:24.785047 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b60cf-de3e-464d-891e-aa0e5f397b1f service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:21:24.797040 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1302/2602] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:21:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:21:32.440050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=32 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:21:34.791073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=37 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:23:23.993031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafe9a9-951f-48d3-8b90-70de70ce663d service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:24.002496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafe9a9-951f-48d3-8b90-70de70ce663d service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:24.006745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07056c-c661-4cc5-a211-d3246068c3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:24.007398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07056c-c661-4cc5-a211-d3246068c3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:24.007878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07056c-c661-4cc5-a211-d3246068c3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:24.008155 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07056c-c661-4cc5-a211-d3246068c3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:24.010100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7d07056c-c661-4cc5-a211-d3246068c3a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:24.127930 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07056c-c661-4cc5-a211-d3246068c3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:24.129353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07056c-c661-4cc5-a211-d3246068c3a1 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:24.149528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1301/2603] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:23:25.697406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b60cf-de3e-464d-891e-aa0e5f397b1f service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:25.699073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0d5b60cf-de3e-464d-891e-aa0e5f397b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:25.846355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b60cf-de3e-464d-891e-aa0e5f397b1f service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:25.851509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea766a41-f616-4f82-86dd-4daf5ab7061b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:25.852162 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea766a41-f616-4f82-86dd-4daf5ab7061b service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:25.852444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea766a41-f616-4f82-86dd-4daf5ab7061b service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:25.852718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea766a41-f616-4f82-86dd-4daf5ab7061b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:25.854692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ea766a41-f616-4f82-86dd-4daf5ab7061b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:25.867289 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea766a41-f616-4f82-86dd-4daf5ab7061b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:25.868827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea766a41-f616-4f82-86dd-4daf5ab7061b service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:25.880463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1303/2604] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:23:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:23:27.682624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71424b-d2a7-48e0-94bf-052b0a842ac0 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:27.683180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71424b-d2a7-48e0-94bf-052b0a842ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:27.683180 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71424b-d2a7-48e0-94bf-052b0a842ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:27.744688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7a71424b-d2a7-48e0-94bf-052b0a842ac0 None None] MFA Rules not processed for user `122fdcf8f13348cf858555a565e5cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:23:27.745927 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7a71424b-d2a7-48e0-94bf-052b0a842ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:27.817441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=100 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:23:27.819284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea766a41-f616-4f82-86dd-4daf5ab7061b service placement] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:27.821465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1302/2605] 10.222.0.45 () {58 vars in 967 bytes} [Tue May 24 13:23:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:23:27.833818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea766a41-f616-4f82-86dd-4daf5ab7061b service placement] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:27.839329 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e6738-d1d0-4c85-ad3a-35527ad5fd0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:27.839977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e6738-d1d0-4c85-ad3a-35527ad5fd0d service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:27.840259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e6738-d1d0-4c85-ad3a-35527ad5fd0d service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:27.840672 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e6738-d1d0-4c85-ad3a-35527ad5fd0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:27.842833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a46e6738-d1d0-4c85-ad3a-35527ad5fd0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:27.858986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e6738-d1d0-4c85-ad3a-35527ad5fd0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:27.861173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e6738-d1d0-4c85-ad3a-35527ad5fd0d service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:27.878509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1304/2606] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:23:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:23:27.879698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71424b-d2a7-48e0-94bf-052b0a842ac0 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:27.881198 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7a71424b-d2a7-48e0-94bf-052b0a842ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:27.883438 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e6738-d1d0-4c85-ad3a-35527ad5fd0d service placement] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:27.885269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a46e6738-d1d0-4c85-ad3a-35527ad5fd0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:27.902416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71424b-d2a7-48e0-94bf-052b0a842ac0 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:27.910014 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:27.911210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:27.911730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:27.912413 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] PATH_INFO: `/v3/users/f7b0dfb3d6514817aba4ab34fd972ba1` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:27.913836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7b0dfb3d6514817aba4ab34fd972ba1)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:27.916356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:27.978236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7b0dfb3d6514817aba4ab34fd972ba1'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:27.988806 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:23:27.990600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7b0dfb3d6514817aba4ab34fd972ba1'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:27.994022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e6738-d1d0-4c85-ad3a-35527ad5fd0d service placement] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:27.998613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7b0dfb3d6514817aba4ab34fd972ba1'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:27.999238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7b0dfb3d6514817aba4ab34fd972ba1'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:28.000227 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2604b27-aaab-4129-8552-f008581603b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00b728319304b1e9ba2f1ee93022187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:28.000879 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2604b27-aaab-4129-8552-f008581603b0 service swift] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.001660 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2604b27-aaab-4129-8552-f008581603b0 service swift] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.001964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2604b27-aaab-4129-8552-f008581603b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.003821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e2604b27-aaab-4129-8552-f008581603b0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:28.007258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7b0dfb3d6514817aba4ab34fd972ba1'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:28.014290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1303/2607] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:23:27 2022] DELETE /identity/v3/users/f7b0dfb3d6514817aba4ab34fd972ba1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:23:28.014942 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2604b27-aaab-4129-8552-f008581603b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:28.016574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2604b27-aaab-4129-8552-f008581603b0 service swift] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:28.026384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1305/2608] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 13:23:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:23:28.205033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:28.208617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:28.292298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd21aac-5785-47bf-99de-47c437653a3b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:28.296995 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:28.297866 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.298727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.299007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] PATH_INFO: `/v3/projects/e57720cd453848a89315ad87ace38161` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.300232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e57720cd453848a89315ad87ace38161)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:28.302152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:28.356826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e57720cd453848a89315ad87ace38161', 'request_id': 'req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:28.366543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e57720cd453848a89315ad87ace38161', 'request_id': 'req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:28.373371 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e57720cd453848a89315ad87ace38161', 'request_id': 'req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:28.389088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] The token cache is being invalidate because project e57720cd453848a89315ad87ace38161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:23:28.389377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:28.390182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:28.392166 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1304/2609] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:23:28 2022] DELETE /identity/v3/projects/e57720cd453848a89315ad87ace38161 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:23:28.399145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb14ee-579a-49d6-a259-dff5a594942d None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.404277 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb14ee-579a-49d6-a259-dff5a594942d None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.405218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb14ee-579a-49d6-a259-dff5a594942d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.481820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9ecb14ee-579a-49d6-a259-dff5a594942d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:23:28.483021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9ecb14ee-579a-49d6-a259-dff5a594942d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:28.544706 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:28.546857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:28.554238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:23:28.568092 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1306/2610] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:23:28.575652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbcb2e-7eb4-4773-83d7-e490f877f16d None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.575944 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbcb2e-7eb4-4773-83d7-e490f877f16d None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.576221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbcb2e-7eb4-4773-83d7-e490f877f16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.593685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9adbcb2e-7eb4-4773-83d7-e490f877f16d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:23:28.595432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9adbcb2e-7eb4-4773-83d7-e490f877f16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:28.603875 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1307/2611] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:23:28.607969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbcb2e-7eb4-4773-83d7-e490f877f16d None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:28.611499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fefc4-03a0-41f0-9a70-c8dac7f71de3 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:28.615121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b62d4-84df-40e0-bf3f-80a4cfe583a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:28.615774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b62d4-84df-40e0-bf3f-80a4cfe583a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.616058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b62d4-84df-40e0-bf3f-80a4cfe583a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.616335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b62d4-84df-40e0-bf3f-80a4cfe583a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.617684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbcb2e-7eb4-4773-83d7-e490f877f16d None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:28.618165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4b3b62d4-84df-40e0-bf3f-80a4cfe583a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:28.622253 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff73c53-66f5-446b-b621-91e8ba59ad9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:28.622941 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff73c53-66f5-446b-b621-91e8ba59ad9f admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.623221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff73c53-66f5-446b-b621-91e8ba59ad9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.623492 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff73c53-66f5-446b-b621-91e8ba59ad9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.624205 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff73c53-66f5-446b-b621-91e8ba59ad9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:23:28.624558 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff73c53-66f5-446b-b621-91e8ba59ad9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:28.626154 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff73c53-66f5-446b-b621-91e8ba59ad9f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:28.630277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b62d4-84df-40e0-bf3f-80a4cfe583a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:28.631717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b62d4-84df-40e0-bf3f-80a4cfe583a5 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:28.638296 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1308/2612] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:23:28.641254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1305/2613] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:23:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:23:28.642668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff73c53-66f5-446b-b621-91e8ba59ad9f admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:28.652269 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff73c53-66f5-446b-b621-91e8ba59ad9f admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:28.656860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd1ea-d077-4238-878c-52b4c4cf9602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:28.658033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd1ea-d077-4238-878c-52b4c4cf9602 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.658379 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd1ea-d077-4238-878c-52b4c4cf9602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.658631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd1ea-d077-4238-878c-52b4c4cf9602 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.659358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcd1ea-d077-4238-878c-52b4c4cf9602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:28.661301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcd1ea-d077-4238-878c-52b4c4cf9602 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:28.698761 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1309/2614] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:23:28 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:23:28.703370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b62d4-84df-40e0-bf3f-80a4cfe583a5 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:28.713192 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b62d4-84df-40e0-bf3f-80a4cfe583a5 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:28.770353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a48e1-539a-49c6-b0f3-b48a266420c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:28.771030 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a48e1-539a-49c6-b0f3-b48a266420c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.771443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a48e1-539a-49c6-b0f3-b48a266420c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.771725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a48e1-539a-49c6-b0f3-b48a266420c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.772418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a48e1-539a-49c6-b0f3-b48a266420c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:28.774836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a48e1-539a-49c6-b0f3-b48a266420c4 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:28.797593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1306/2615] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:23:28 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:23:28.802410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd1ea-d077-4238-878c-52b4c4cf9602 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:28.812617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd1ea-d077-4238-878c-52b4c4cf9602 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:28.817314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843a1c7-d280-4f3e-be2c-5ffca00dc4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:28.818043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a1c7-d280-4f3e-be2c-5ffca00dc4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.818427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a1c7-d280-4f3e-be2c-5ffca00dc4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.818772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a1c7-d280-4f3e-be2c-5ffca00dc4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.819598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843a1c7-d280-4f3e-be2c-5ffca00dc4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:28.821288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843a1c7-d280-4f3e-be2c-5ffca00dc4e5 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:28.834902 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1310/2616] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:23:28.835899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a48e1-539a-49c6-b0f3-b48a266420c4 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:28.846451 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a48e1-539a-49c6-b0f3-b48a266420c4 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:28.851410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aaecdd-b83c-4982-b739-661b0ca5efb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:28.852292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaecdd-b83c-4982-b739-661b0ca5efb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.852577 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaecdd-b83c-4982-b739-661b0ca5efb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.852858 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaecdd-b83c-4982-b739-661b0ca5efb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.853620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aaecdd-b83c-4982-b739-661b0ca5efb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:28.855255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aaecdd-b83c-4982-b739-661b0ca5efb2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:28.869231 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1307/2617] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:23:28.870257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843a1c7-d280-4f3e-be2c-5ffca00dc4e5 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:28.881123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843a1c7-d280-4f3e-be2c-5ffca00dc4e5 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:28.885748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ef79a-54c1-4200-8143-1fa6561dae8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:28.886661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ef79a-54c1-4200-8143-1fa6561dae8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.886947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ef79a-54c1-4200-8143-1fa6561dae8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.887223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ef79a-54c1-4200-8143-1fa6561dae8a admin admin] PATH_INFO: `/v3/projects/cb54f85be3994a40a4b52e576b36c6da/users/e13a5203bc06439ba86befd2eb849ba5/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.902450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ef79a-54c1-4200-8143-1fa6561dae8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb54f85be3994a40a4b52e576b36c6da, user_id=e13a5203bc06439ba86befd2eb849ba5, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:28.905120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ef79a-54c1-4200-8143-1fa6561dae8a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:28.919143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1311/2618] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:23:28 2022] PUT /identity/v3/projects/cb54f85be3994a40a4b52e576b36c6da/users/e13a5203bc06439ba86befd2eb849ba5/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:23:28.924700 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd8dab-c7d9-484a-b63d-841c770961cc None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:28.924987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd8dab-c7d9-484a-b63d-841c770961cc None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:28.925264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd8dab-c7d9-484a-b63d-841c770961cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:28.943403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-dcdd8dab-c7d9-484a-b63d-841c770961cc None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:23:28.944607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-dcdd8dab-c7d9-484a-b63d-841c770961cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:29.004932 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1308/2619] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:23:29.016781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ef79a-54c1-4200-8143-1fa6561dae8a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:29.058821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:23:29.086807 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ef79a-54c1-4200-8143-1fa6561dae8a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:29.090604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a5880-fc14-4a7d-8121-9c57db5dd2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:29.091319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a5880-fc14-4a7d-8121-9c57db5dd2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:29.091608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a5880-fc14-4a7d-8121-9c57db5dd2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:29.091882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a5880-fc14-4a7d-8121-9c57db5dd2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:29.101143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a5880-fc14-4a7d-8121-9c57db5dd2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:29.102621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a5880-fc14-4a7d-8121-9c57db5dd2e6 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:29.113202 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1312/2620] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:23:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:23:34.652308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c0466-45d8-4f37-a482-814be187505d None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:34.652842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c0466-45d8-4f37-a482-814be187505d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:34.653253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c0466-45d8-4f37-a482-814be187505d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:34.694645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-432c0466-45d8-4f37-a482-814be187505d None None] MFA Rules not processed for user `e13a5203bc06439ba86befd2eb849ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:23:34.696045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-432c0466-45d8-4f37-a482-814be187505d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:34.720673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:23:34.790394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1309/2621] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:23:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:23:34.805637 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a5880-fc14-4a7d-8121-9c57db5dd2e6 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:34.807227 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b52a5880-fc14-4a7d-8121-9c57db5dd2e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:34.899284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a5880-fc14-4a7d-8121-9c57db5dd2e6 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:34.905971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bbab5-3dc2-4887-9f25-b85b7e4d3547 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:34.906916 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bbab5-3dc2-4887-9f25-b85b7e4d3547 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:34.907966 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bbab5-3dc2-4887-9f25-b85b7e4d3547 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:34.908371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bbab5-3dc2-4887-9f25-b85b7e4d3547 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:34.922562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bbab5-3dc2-4887-9f25-b85b7e4d3547 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:34.924795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bbab5-3dc2-4887-9f25-b85b7e4d3547 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:34.936119 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1313/2622] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:23:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:23:37.065535 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c0466-45d8-4f37-a482-814be187505d None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:37.067698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-432c0466-45d8-4f37-a482-814be187505d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:37.114552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bbab5-3dc2-4887-9f25-b85b7e4d3547 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:37.114552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-660bbab5-3dc2-4887-9f25-b85b7e4d3547 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:37.151250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bbab5-3dc2-4887-9f25-b85b7e4d3547 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:37.158358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:37.159364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:37.159696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:37.160113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] PATH_INFO: `/v3/users/66b0a0ef7ea24caaa9540786a4d28682` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:37.162834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66b0a0ef7ea24caaa9540786a4d28682)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:37.165861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:37.232254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c0466-45d8-4f37-a482-814be187505d None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:37.240315 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ab8f6-a5d8-43eb-81ef-3462e360adec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:37.241228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ab8f6-a5d8-43eb-81ef-3462e360adec service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:37.241580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ab8f6-a5d8-43eb-81ef-3462e360adec service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:37.242105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ab8f6-a5d8-43eb-81ef-3462e360adec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:37.244632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-0b2ab8f6-a5d8-43eb-81ef-3462e360adec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:37.260928 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ab8f6-a5d8-43eb-81ef-3462e360adec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:37.262893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ab8f6-a5d8-43eb-81ef-3462e360adec service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:37.280260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1310/2623] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:23:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:23:37.281853 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d1d52-c6e4-4dad-8ceb-402bc2700639 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:37.282900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d1d52-c6e4-4dad-8ceb-402bc2700639 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:37.283450 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d1d52-c6e4-4dad-8ceb-402bc2700639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:37.294925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66b0a0ef7ea24caaa9540786a4d28682'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.314934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66b0a0ef7ea24caaa9540786a4d28682'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.316749 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-656d1d52-c6e4-4dad-8ceb-402bc2700639 None None] MFA Rules not processed for user `e13a5203bc06439ba86befd2eb849ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:23:37.318411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-656d1d52-c6e4-4dad-8ceb-402bc2700639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:37.319466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66b0a0ef7ea24caaa9540786a4d28682'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.321896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66b0a0ef7ea24caaa9540786a4d28682'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.336169 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1311/2624] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:23:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:23:37.342510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66b0a0ef7ea24caaa9540786a4d28682'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.365828 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1314/2625] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:23:37 2022] DELETE /identity/v3/users/66b0a0ef7ea24caaa9540786a4d28682 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:23:37.607058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d1d52-c6e4-4dad-8ceb-402bc2700639 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:37.607931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-656d1d52-c6e4-4dad-8ceb-402bc2700639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:37.624288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d1d52-c6e4-4dad-8ceb-402bc2700639 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:37.701992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:37.703276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:37.703767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:37.704819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] PATH_INFO: `/v3/projects/0df8852ede3b4845b824cc9345afbb5d` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:37.707328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df8852ede3b4845b824cc9345afbb5d)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:37.710038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:37.742032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:37.743766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:37.761138 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:23:37.773316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df8852ede3b4845b824cc9345afbb5d', 'request_id': 'req-ca68c390-cb61-4d06-9434-bb2302dce264'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.782022 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df8852ede3b4845b824cc9345afbb5d', 'request_id': 'req-ca68c390-cb61-4d06-9434-bb2302dce264'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.784903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:23:37.791082 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df8852ede3b4845b824cc9345afbb5d', 'request_id': 'req-ca68c390-cb61-4d06-9434-bb2302dce264'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.805464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] The token cache is being invalidate because project 0df8852ede3b4845b824cc9345afbb5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:23:37.805678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.806505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.808094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1312/2626] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:23:37 2022] DELETE /identity/v3/projects/0df8852ede3b4845b824cc9345afbb5d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:23:37.811207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7aab1-cfe3-465c-9837-b2d40b1aa417 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:37.812278 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:37.814457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:37.868934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f040fda-d8da-4b9a-97f1-5fc412acc1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:37.869592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f040fda-d8da-4b9a-97f1-5fc412acc1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:37.869873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f040fda-d8da-4b9a-97f1-5fc412acc1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:37.870146 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f040fda-d8da-4b9a-97f1-5fc412acc1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:37.872066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6f040fda-d8da-4b9a-97f1-5fc412acc1c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:37.882856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f040fda-d8da-4b9a-97f1-5fc412acc1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:37.884498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f040fda-d8da-4b9a-97f1-5fc412acc1c0 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:37.885533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68c390-cb61-4d06-9434-bb2302dce264 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:37.890245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:37.891242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:37.891634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:37.891982 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] PATH_INFO: `/v3/users/d165c1a66ad245fcb611d96a290b6162` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:37.894321 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1315/2627] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:23:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:23:37.909483 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d165c1a66ad245fcb611d96a290b6162)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:37.911467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:37.971062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd165c1a66ad245fcb611d96a290b6162'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.987529 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd165c1a66ad245fcb611d96a290b6162'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:37.999158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd165c1a66ad245fcb611d96a290b6162'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:38.000184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd165c1a66ad245fcb611d96a290b6162'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:38.012869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd165c1a66ad245fcb611d96a290b6162'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:38.022401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1313/2628] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:23:37 2022] DELETE /identity/v3/users/d165c1a66ad245fcb611d96a290b6162 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:23:38.225316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f040fda-d8da-4b9a-97f1-5fc412acc1c0 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:38.239817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f040fda-d8da-4b9a-97f1-5fc412acc1c0 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:38.287329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:38.289418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:38.344276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:38.349906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:38.359212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:38.359212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] PATH_INFO: `/v3/projects/a067c6767fc848b39d7ab93777cdc1b3` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:38.367448 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a067c6767fc848b39d7ab93777cdc1b3)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:38.369918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:38.382886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:23:38.435663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adecfff-8771-405f-afbc-4104532b9ec2 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:38.445653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687dd0c4-086d-4c0d-a748-97a62eb0f10c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:38.445653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dd0c4-086d-4c0d-a748-97a62eb0f10c service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:38.445653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dd0c4-086d-4c0d-a748-97a62eb0f10c service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:38.445653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dd0c4-086d-4c0d-a748-97a62eb0f10c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:38.447647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-687dd0c4-086d-4c0d-a748-97a62eb0f10c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:38.459056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a067c6767fc848b39d7ab93777cdc1b3', 'request_id': 'req-780e3d70-cf40-44c1-b4f1-a9af9de251a6'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:38.461773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687dd0c4-086d-4c0d-a748-97a62eb0f10c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:38.464320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687dd0c4-086d-4c0d-a748-97a62eb0f10c service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:38.477083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a067c6767fc848b39d7ab93777cdc1b3', 'request_id': 'req-780e3d70-cf40-44c1-b4f1-a9af9de251a6'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:38.477747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1314/2629] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:23:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:23:38.504738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a067c6767fc848b39d7ab93777cdc1b3', 'request_id': 'req-780e3d70-cf40-44c1-b4f1-a9af9de251a6'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:38.518751 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] The token cache is being invalidate because project a067c6767fc848b39d7ab93777cdc1b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:23:38.519130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:38.519815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:23:38.521696 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1316/2630] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:23:38 2022] DELETE /identity/v3/projects/a067c6767fc848b39d7ab93777cdc1b3 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:23:40.917709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687dd0c4-086d-4c0d-a748-97a62eb0f10c service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:40.920052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-687dd0c4-086d-4c0d-a748-97a62eb0f10c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:41.018157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687dd0c4-086d-4c0d-a748-97a62eb0f10c service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:41.090127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b9026-a1db-4310-8b95-88a2efe984ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:41.090906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9026-a1db-4310-8b95-88a2efe984ca service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:41.091197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9026-a1db-4310-8b95-88a2efe984ca service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:41.091478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9026-a1db-4310-8b95-88a2efe984ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:41.093269 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-504b9026-a1db-4310-8b95-88a2efe984ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:41.106156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b9026-a1db-4310-8b95-88a2efe984ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:41.107869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b9026-a1db-4310-8b95-88a2efe984ca service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:41.120445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1315/2631] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:23:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:23:47.198167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:23:47.200211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:47.219099 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e3d70-cf40-44c1-b4f1-a9af9de251a6 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:23:47.224739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2ee67-53a0-4a1f-ab0d-06d5559c0ab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:23:47.225396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2ee67-53a0-4a1f-ab0d-06d5559c0ab7 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:23:47.225681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2ee67-53a0-4a1f-ab0d-06d5559c0ab7 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:23:47.225958 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2ee67-53a0-4a1f-ab0d-06d5559c0ab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:23:47.227771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-eae2ee67-53a0-4a1f-ab0d-06d5559c0ab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:23:47.243081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2ee67-53a0-4a1f-ab0d-06d5559c0ab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:23:47.244693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2ee67-53a0-4a1f-ab0d-06d5559c0ab7 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:23:47.251293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:23:47.273393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1317/2632] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:23:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:23:51.116060 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=78 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:24:22.271606 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b9026-a1db-4310-8b95-88a2efe984ca service nova] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:24:22.293105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b9026-a1db-4310-8b95-88a2efe984ca service nova] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:24:22.298544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2199e20-797f-48e0-9dc7-baba3ec1e50a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:24:22.299185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2199e20-797f-48e0-9dc7-baba3ec1e50a service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:24:22.299469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2199e20-797f-48e0-9dc7-baba3ec1e50a service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:24:22.299746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2199e20-797f-48e0-9dc7-baba3ec1e50a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:24:22.301496 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e2199e20-797f-48e0-9dc7-baba3ec1e50a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:24:22.368111 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2199e20-797f-48e0-9dc7-baba3ec1e50a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:24:22.369961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2199e20-797f-48e0-9dc7-baba3ec1e50a service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:24:22.389080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1316/2633] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:24:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:24:32.386415 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=41 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:26:22.919900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2ee67-53a0-4a1f-ab0d-06d5559c0ab7 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:26:22.921293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-eae2ee67-53a0-4a1f-ab0d-06d5559c0ab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:26:22.995300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2ee67-53a0-4a1f-ab0d-06d5559c0ab7 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:26:22.998922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4f5b9-6334-4292-8d43-18721a6eff92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:26:22.999568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4f5b9-6334-4292-8d43-18721a6eff92 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:26:22.999848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4f5b9-6334-4292-8d43-18721a6eff92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:26:23.000119 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4f5b9-6334-4292-8d43-18721a6eff92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:26:23.001865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-94b4f5b9-6334-4292-8d43-18721a6eff92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:26:23.021455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4f5b9-6334-4292-8d43-18721a6eff92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:26:23.022923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4f5b9-6334-4292-8d43-18721a6eff92 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:26:23.043177 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1318/2634] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:26:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:26:25.893560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2199e20-797f-48e0-9dc7-baba3ec1e50a service nova] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:26:25.894987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e2199e20-797f-48e0-9dc7-baba3ec1e50a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:26:25.906565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2199e20-797f-48e0-9dc7-baba3ec1e50a service nova] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:26:25.910059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17346588-03fa-491b-a551-2ac3e253f7be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:26:25.910750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17346588-03fa-491b-a551-2ac3e253f7be service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:26:25.911035 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17346588-03fa-491b-a551-2ac3e253f7be service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:26:25.911311 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17346588-03fa-491b-a551-2ac3e253f7be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:26:25.913044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-17346588-03fa-491b-a551-2ac3e253f7be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:26:25.922677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17346588-03fa-491b-a551-2ac3e253f7be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:26:25.924093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17346588-03fa-491b-a551-2ac3e253f7be service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:26:25.931836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1317/2635] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:26:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:26:26.340128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4f5b9-6334-4292-8d43-18721a6eff92 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:26:26.349298 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4f5b9-6334-4292-8d43-18721a6eff92 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:26:26.352862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec434a6-6945-44bd-9b05-f5ce1b15672a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:26:26.353498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec434a6-6945-44bd-9b05-f5ce1b15672a service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:26:26.353783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec434a6-6945-44bd-9b05-f5ce1b15672a service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:26:26.354061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec434a6-6945-44bd-9b05-f5ce1b15672a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:26:26.355839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-dec434a6-6945-44bd-9b05-f5ce1b15672a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:26:26.369948 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec434a6-6945-44bd-9b05-f5ce1b15672a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:26:26.371425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec434a6-6945-44bd-9b05-f5ce1b15672a service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:26:26.382530 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1319/2636] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:26:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:26:30.837059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17346588-03fa-491b-a551-2ac3e253f7be service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:26:30.838632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-17346588-03fa-491b-a551-2ac3e253f7be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:26:30.913805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17346588-03fa-491b-a551-2ac3e253f7be service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:26:30.917552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c118d9-3e85-44ea-b5a1-313b2287c20b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:26:30.918212 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c118d9-3e85-44ea-b5a1-313b2287c20b service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:26:30.918539 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c118d9-3e85-44ea-b5a1-313b2287c20b service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:26:30.918826 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c118d9-3e85-44ea-b5a1-313b2287c20b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:26:30.920660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-87c118d9-3e85-44ea-b5a1-313b2287c20b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:26:30.980347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c118d9-3e85-44ea-b5a1-313b2287c20b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:26:30.982059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c118d9-3e85-44ea-b5a1-313b2287c20b service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:26:30.991977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1318/2637] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:26:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:26:36.377517 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=48 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:26:40.987544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=58 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:28:28.148832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec434a6-6945-44bd-9b05-f5ce1b15672a service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:28:28.150254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-dec434a6-6945-44bd-9b05-f5ce1b15672a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:28:28.217720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec434a6-6945-44bd-9b05-f5ce1b15672a service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:28:28.222763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111664e-f1bf-4434-91a4-8d90bf4d4384 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:28:28.223411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111664e-f1bf-4434-91a4-8d90bf4d4384 service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:28:28.223698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111664e-f1bf-4434-91a4-8d90bf4d4384 service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:28:28.223978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111664e-f1bf-4434-91a4-8d90bf4d4384 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:28:28.225732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8111664e-f1bf-4434-91a4-8d90bf4d4384 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:28:28.237744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111664e-f1bf-4434-91a4-8d90bf4d4384 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:28:28.239357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111664e-f1bf-4434-91a4-8d90bf4d4384 service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:28:28.249755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1320/2638] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:28:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:28:38.244971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=32 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:29:24.407015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c118d9-3e85-44ea-b5a1-313b2287c20b service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:29:24.409220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-87c118d9-3e85-44ea-b5a1-313b2287c20b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:29:24.423196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c118d9-3e85-44ea-b5a1-313b2287c20b service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:29:24.426954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e706363-d27c-48a5-b25f-4faa2e612a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:29:24.427692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e706363-d27c-48a5-b25f-4faa2e612a0a service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:29:24.428083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e706363-d27c-48a5-b25f-4faa2e612a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:29:24.428249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e706363-d27c-48a5-b25f-4faa2e612a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:29:24.437102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e706363-d27c-48a5-b25f-4faa2e612a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:29:24.439078 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e706363-d27c-48a5-b25f-4faa2e612a0a service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:29:24.448326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1319/2639] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:29:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:29:24.497388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111664e-f1bf-4434-91a4-8d90bf4d4384 service placement] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:29:24.507104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111664e-f1bf-4434-91a4-8d90bf4d4384 service placement] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:29:24.510647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf24105-79c6-49e2-bbbc-0f42fed0d597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:29:24.511280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf24105-79c6-49e2-bbbc-0f42fed0d597 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:29:24.511562 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf24105-79c6-49e2-bbbc-0f42fed0d597 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:29:24.511835 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf24105-79c6-49e2-bbbc-0f42fed0d597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:29:24.513613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0cf24105-79c6-49e2-bbbc-0f42fed0d597 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:29:24.571029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf24105-79c6-49e2-bbbc-0f42fed0d597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:29:24.572509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf24105-79c6-49e2-bbbc-0f42fed0d597 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:29:24.582124 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1321/2640] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:29:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:29:34.443949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:29:34.577360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=28 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:31:30.587102 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e706363-d27c-48a5-b25f-4faa2e612a0a service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:31:30.599628 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e706363-d27c-48a5-b25f-4faa2e612a0a service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:31:30.605318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f73d8-c03e-4db2-bbb0-44177aacedab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:31:30.605961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f73d8-c03e-4db2-bbb0-44177aacedab service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:31:30.606246 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f73d8-c03e-4db2-bbb0-44177aacedab service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:31:30.606565 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f73d8-c03e-4db2-bbb0-44177aacedab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:31:30.615304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f73d8-c03e-4db2-bbb0-44177aacedab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:31:30.616870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f73d8-c03e-4db2-bbb0-44177aacedab service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:31:30.644012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1320/2641] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:31:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:31:30.928808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf24105-79c6-49e2-bbbc-0f42fed0d597 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:31:30.938960 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf24105-79c6-49e2-bbbc-0f42fed0d597 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:31:30.942507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12432bfe-cb4a-4669-abdb-f0d99860a68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:31:30.943143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12432bfe-cb4a-4669-abdb-f0d99860a68e service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:31:30.943422 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12432bfe-cb4a-4669-abdb-f0d99860a68e service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:31:30.943691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12432bfe-cb4a-4669-abdb-f0d99860a68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:31:30.951143 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12432bfe-cb4a-4669-abdb-f0d99860a68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:31:30.952577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12432bfe-cb4a-4669-abdb-f0d99860a68e service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:31:30.960303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1322/2642] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:31:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:31:40.641436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=13 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:31:40.956802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=6 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:32:31.575187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f73d8-c03e-4db2-bbb0-44177aacedab service nova] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:32:31.584283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f73d8-c03e-4db2-bbb0-44177aacedab service nova] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:32:31.587860 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41cd01-2680-4cd8-b832-3203640fd56e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:32:31.588493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41cd01-2680-4cd8-b832-3203640fd56e service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:32:31.588786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41cd01-2680-4cd8-b832-3203640fd56e service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:32:31.589063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41cd01-2680-4cd8-b832-3203640fd56e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:32:31.599898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41cd01-2680-4cd8-b832-3203640fd56e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:32:31.601323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41cd01-2680-4cd8-b832-3203640fd56e service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:32:31.612829 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1321/2643] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:32:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:32:33.174958 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12432bfe-cb4a-4669-abdb-f0d99860a68e service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:32:33.189114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12432bfe-cb4a-4669-abdb-f0d99860a68e service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:32:33.195101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df636122-c574-4ffa-bfad-bd45d71710cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:32:33.196212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636122-c574-4ffa-bfad-bd45d71710cf service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:32:33.196688 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636122-c574-4ffa-bfad-bd45d71710cf service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:32:33.196967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636122-c574-4ffa-bfad-bd45d71710cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:32:33.208091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df636122-c574-4ffa-bfad-bd45d71710cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:32:33.212090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df636122-c574-4ffa-bfad-bd45d71710cf service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:32:33.240154 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1323/2644] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:32:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:32:36.116596 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b549b-3805-4d28-8633-f473353217ed None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:32:36.117108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b549b-3805-4d28-8633-f473353217ed None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:32:36.117221 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b549b-3805-4d28-8633-f473353217ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:32:36.135240 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-975b549b-3805-4d28-8633-f473353217ed None None] MFA Rules not processed for user `e13a5203bc06439ba86befd2eb849ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:32:36.136434 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-975b549b-3805-4d28-8633-f473353217ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:32:36.143600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1322/2645] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:32:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:32:36.435163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df636122-c574-4ffa-bfad-bd45d71710cf service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:32:36.444399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df636122-c574-4ffa-bfad-bd45d71710cf service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:32:36.447802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d0612-ab38-46f6-a1c5-a140276c1be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:32:36.448429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d0612-ab38-46f6-a1c5-a140276c1be6 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:32:36.448710 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d0612-ab38-46f6-a1c5-a140276c1be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:32:36.448983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d0612-ab38-46f6-a1c5-a140276c1be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:32:36.451227 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8b5d0612-ab38-46f6-a1c5-a140276c1be6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:32:36.473157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d0612-ab38-46f6-a1c5-a140276c1be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:32:36.474869 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d0612-ab38-46f6-a1c5-a140276c1be6 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:32:36.483326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1324/2646] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:32:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:32:36.803283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b549b-3805-4d28-8633-f473353217ed None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:32:36.804699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-975b549b-3805-4d28-8633-f473353217ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:32:36.806685 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d0612-ab38-46f6-a1c5-a140276c1be6 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:32:36.816503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d0612-ab38-46f6-a1c5-a140276c1be6 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:32:36.819656 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b549b-3805-4d28-8633-f473353217ed None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:32:36.820338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62add1-952a-4938-b7ba-8dc42c66e9ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:32:36.820972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62add1-952a-4938-b7ba-8dc42c66e9ec service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:32:36.821252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62add1-952a-4938-b7ba-8dc42c66e9ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:32:36.821904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62add1-952a-4938-b7ba-8dc42c66e9ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:32:36.824709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c4ca6-9563-4e19-a73c-c9780b194351 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:32:36.825351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c4ca6-9563-4e19-a73c-c9780b194351 service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:32:36.825641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c4ca6-9563-4e19-a73c-c9780b194351 service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:32:36.825922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c4ca6-9563-4e19-a73c-c9780b194351 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:32:36.827697 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-231c4ca6-9563-4e19-a73c-c9780b194351 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:32:36.832185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62add1-952a-4938-b7ba-8dc42c66e9ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:32:36.833604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62add1-952a-4938-b7ba-8dc42c66e9ec service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:32:36.839487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c4ca6-9563-4e19-a73c-c9780b194351 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:32:36.841774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c4ca6-9563-4e19-a73c-c9780b194351 service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:32:36.845318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1325/2647] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:32:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:32:36.857009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1323/2648] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:32:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:32:39.124857 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62add1-952a-4938-b7ba-8dc42c66e9ec service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:32:39.127213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7a62add1-952a-4938-b7ba-8dc42c66e9ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:32:39.167016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62add1-952a-4938-b7ba-8dc42c66e9ec service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:32:39.178474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b980-19ea-4cd9-a8b7-75c1dea177f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:32:39.184593 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b980-19ea-4cd9-a8b7-75c1dea177f2 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:32:39.185116 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b980-19ea-4cd9-a8b7-75c1dea177f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:32:39.185575 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b980-19ea-4cd9-a8b7-75c1dea177f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:32:39.187746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3a04b980-19ea-4cd9-a8b7-75c1dea177f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:32:39.202796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04b980-19ea-4cd9-a8b7-75c1dea177f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:32:39.205069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04b980-19ea-4cd9-a8b7-75c1dea177f2 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:32:39.217649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1326/2649] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:32:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:32:45.748838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c4ca6-9563-4e19-a73c-c9780b194351 service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:32:45.762133 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c4ca6-9563-4e19-a73c-c9780b194351 service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:32:45.767704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f88a8-c84e-4619-a604-667334df07dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:32:45.768350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f88a8-c84e-4619-a604-667334df07dc service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:32:45.768637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f88a8-c84e-4619-a604-667334df07dc service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:32:45.768917 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f88a8-c84e-4619-a604-667334df07dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:32:45.778505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f88a8-c84e-4619-a604-667334df07dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:32:45.780129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f88a8-c84e-4619-a604-667334df07dc service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:32:45.791199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1324/2650] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:32:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:32:49.212130 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=34 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:32:55.786426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=21 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:33:24.938980 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b980-19ea-4cd9-a8b7-75c1dea177f2 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:33:24.948538 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b980-19ea-4cd9-a8b7-75c1dea177f2 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:33:24.952120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6e754-1526-4c2a-9d87-7a41a1507c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:33:24.952763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6e754-1526-4c2a-9d87-7a41a1507c80 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:33:24.953045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6e754-1526-4c2a-9d87-7a41a1507c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:33:24.953322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6e754-1526-4c2a-9d87-7a41a1507c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:33:24.955109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-76f6e754-1526-4c2a-9d87-7a41a1507c80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:33:25.042011 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6e754-1526-4c2a-9d87-7a41a1507c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:33:25.043512 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6e754-1526-4c2a-9d87-7a41a1507c80 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:33:25.062206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1327/2651] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:33:28.029952 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f88a8-c84e-4619-a604-667334df07dc service nova] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:33:28.089051 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f88a8-c84e-4619-a604-667334df07dc service nova] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:33:28.094805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f95b7-2c7d-439b-af57-c9ca6fabc459 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:33:28.095613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f95b7-2c7d-439b-af57-c9ca6fabc459 service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:33:28.095897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f95b7-2c7d-439b-af57-c9ca6fabc459 service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:33:28.096245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f95b7-2c7d-439b-af57-c9ca6fabc459 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:33:28.098690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-769f95b7-2c7d-439b-af57-c9ca6fabc459 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:33:28.137308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f95b7-2c7d-439b-af57-c9ca6fabc459 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:33:28.138946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f95b7-2c7d-439b-af57-c9ca6fabc459 service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:33:28.149436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1325/2652] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:33:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:33:31.229385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6e754-1526-4c2a-9d87-7a41a1507c80 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:33:31.231185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-76f6e754-1526-4c2a-9d87-7a41a1507c80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:33:31.280073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6e754-1526-4c2a-9d87-7a41a1507c80 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:33:31.284967 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:33:31.285834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:33:31.286148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:33:31.286483 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] PATH_INFO: `/v3/users/d87d9358c42a43fea94927869c44dc82` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:33:31.299312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d87d9358c42a43fea94927869c44dc82)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:33:31.301335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:33:31.348960 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd87d9358c42a43fea94927869c44dc82'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.356299 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd87d9358c42a43fea94927869c44dc82'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.356998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd87d9358c42a43fea94927869c44dc82'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.357616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd87d9358c42a43fea94927869c44dc82'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.369358 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd87d9358c42a43fea94927869c44dc82'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.376900 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:33:31.382137 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1328/2653] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:33:31 2022] DELETE /identity/v3/users/d87d9358c42a43fea94927869c44dc82 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:33:31.549497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f95b7-2c7d-439b-af57-c9ca6fabc459 service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:33:31.561378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f95b7-2c7d-439b-af57-c9ca6fabc459 service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:33:31.630627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:33:31.631501 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:33:31.631786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:33:31.632182 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] PATH_INFO: `/v3/projects/b20159be9d404dd2812ffeffe055ec67` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:33:31.633432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b20159be9d404dd2812ffeffe055ec67)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:33:31.636017 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:33:31.682984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b20159be9d404dd2812ffeffe055ec67', 'request_id': 'req-0b40089d-d466-49be-97cd-ffdeb763b86c'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.690515 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b20159be9d404dd2812ffeffe055ec67', 'request_id': 'req-0b40089d-d466-49be-97cd-ffdeb763b86c'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.696154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b20159be9d404dd2812ffeffe055ec67', 'request_id': 'req-0b40089d-d466-49be-97cd-ffdeb763b86c'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.707870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] The token cache is being invalidate because project b20159be9d404dd2812ffeffe055ec67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:33:31.708158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.708804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.710425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1326/2654] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:33:31 2022] DELETE /identity/v3/projects/b20159be9d404dd2812ffeffe055ec67 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:33:31.714386 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:33:31.715783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:33:31.728941 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5335b85-8dbb-4471-96af-ee4b9c0d08f8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:33:31.783618 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:33:31.784490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:33:31.784779 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:33:31.785057 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] PATH_INFO: `/v3/users/cd55e696b29341f98e31cf36a34f3378` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:33:31.798107 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd55e696b29341f98e31cf36a34f3378)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:33:31.800103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:33:31.845265 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd55e696b29341f98e31cf36a34f3378'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.852537 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd55e696b29341f98e31cf36a34f3378'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.853157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd55e696b29341f98e31cf36a34f3378'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.853820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd55e696b29341f98e31cf36a34f3378'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.865757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd55e696b29341f98e31cf36a34f3378'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:31.878227 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1329/2655] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:33:31 2022] DELETE /identity/v3/users/cd55e696b29341f98e31cf36a34f3378 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:33:32.061530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:33:32.063119 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:33:32.080868 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40089d-d466-49be-97cd-ffdeb763b86c admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:33:32.099484 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:33:32.138926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:33:32.139801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:33:32.140085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:33:32.140359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] PATH_INFO: `/v3/projects/f3fb15f71b4d4ccdb5e72d0804488b7e` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:33:32.150786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3fb15f71b4d4ccdb5e72d0804488b7e)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:33:32.152701 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:33:32.197963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3fb15f71b4d4ccdb5e72d0804488b7e', 'request_id': 'req-d91022d6-1525-4415-b6f7-bb97e72e3d34'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:32.205276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3fb15f71b4d4ccdb5e72d0804488b7e', 'request_id': 'req-d91022d6-1525-4415-b6f7-bb97e72e3d34'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:32.210929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3fb15f71b4d4ccdb5e72d0804488b7e', 'request_id': 'req-d91022d6-1525-4415-b6f7-bb97e72e3d34'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:32.222106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] The token cache is being invalidate because project f3fb15f71b4d4ccdb5e72d0804488b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:33:32.222430 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:32.223316 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:33:32.224844 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1327/2656] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:33:32 2022] DELETE /identity/v3/projects/f3fb15f71b4d4ccdb5e72d0804488b7e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:33:36.170243 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:33:36.171666 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:33:36.239129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b4276-4080-4329-98a1-f67f402eff78 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:33:36.243691 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:33:36.298561 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af27d20-5f4a-41db-b2bd-1fd7dc319c18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:33:36.299207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af27d20-5f4a-41db-b2bd-1fd7dc319c18 service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:33:36.299490 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af27d20-5f4a-41db-b2bd-1fd7dc319c18 service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:33:36.299765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af27d20-5f4a-41db-b2bd-1fd7dc319c18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:33:36.301523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6af27d20-5f4a-41db-b2bd-1fd7dc319c18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:33:36.316246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af27d20-5f4a-41db-b2bd-1fd7dc319c18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:33:36.319931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af27d20-5f4a-41db-b2bd-1fd7dc319c18 service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:33:36.329054 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1330/2657] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:33:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:33:42.219148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:33:46.324340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=28 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:34:24.984551 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:34:24.985965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:34:25.052812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91022d6-1525-4415-b6f7-bb97e72e3d34 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:34:25.056873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a6cf9-50d9-408a-943a-6f016a1de82c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:34:25.057511 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a6cf9-50d9-408a-943a-6f016a1de82c service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:34:25.057796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a6cf9-50d9-408a-943a-6f016a1de82c service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:34:25.058071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a6cf9-50d9-408a-943a-6f016a1de82c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:34:25.059850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d74a6cf9-50d9-408a-943a-6f016a1de82c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:34:25.137290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a6cf9-50d9-408a-943a-6f016a1de82c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:34:25.138778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a6cf9-50d9-408a-943a-6f016a1de82c service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:34:25.156482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1328/2658] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:34:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:34:35.153987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=66 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:36:30.547939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af27d20-5f4a-41db-b2bd-1fd7dc319c18 service placement] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:36:30.549435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6af27d20-5f4a-41db-b2bd-1fd7dc319c18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:36:30.570977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af27d20-5f4a-41db-b2bd-1fd7dc319c18 service placement] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:36:30.574670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3a6e6-cbdc-46da-9a85-df9dfef76dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:36:30.575309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3a6e6-cbdc-46da-9a85-df9dfef76dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:36:30.575597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3a6e6-cbdc-46da-9a85-df9dfef76dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:36:30.575873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3a6e6-cbdc-46da-9a85-df9dfef76dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:36:30.577586 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7ea3a6e6-cbdc-46da-9a85-df9dfef76dc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:36:30.647373 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3a6e6-cbdc-46da-9a85-df9dfef76dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:36:30.648818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3a6e6-cbdc-46da-9a85-df9dfef76dc7 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:36:30.658626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1331/2659] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:36:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:36:38.867657 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a6cf9-50d9-408a-943a-6f016a1de82c service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:36:38.869116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d74a6cf9-50d9-408a-943a-6f016a1de82c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:36:38.964091 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a6cf9-50d9-408a-943a-6f016a1de82c service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:36:38.967707 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5190289-81d9-4a16-bfe2-edb6a526c729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:36:38.968356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5190289-81d9-4a16-bfe2-edb6a526c729 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:36:38.968639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5190289-81d9-4a16-bfe2-edb6a526c729 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:36:38.969172 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5190289-81d9-4a16-bfe2-edb6a526c729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:36:38.970966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e5190289-81d9-4a16-bfe2-edb6a526c729 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:36:38.980264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5190289-81d9-4a16-bfe2-edb6a526c729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:36:38.981679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5190289-81d9-4a16-bfe2-edb6a526c729 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:36:38.989520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1329/2660] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:36:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:36:40.654458 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=38 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:36:48.985958 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=35 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:38:41.126251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3a6e6-cbdc-46da-9a85-df9dfef76dc7 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:38:41.127651 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7ea3a6e6-cbdc-46da-9a85-df9dfef76dc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:38:41.141754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3a6e6-cbdc-46da-9a85-df9dfef76dc7 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:38:41.146608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397245b-9182-48ae-a6b8-f4b5f0c9e86a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:38:41.147239 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397245b-9182-48ae-a6b8-f4b5f0c9e86a service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:38:41.147519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397245b-9182-48ae-a6b8-f4b5f0c9e86a service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:38:41.147791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397245b-9182-48ae-a6b8-f4b5f0c9e86a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:38:41.156244 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397245b-9182-48ae-a6b8-f4b5f0c9e86a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:38:41.157947 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397245b-9182-48ae-a6b8-f4b5f0c9e86a service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:38:41.168215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1332/2661] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:38:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:38:51.163460 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=6 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:39:24.906323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5190289-81d9-4a16-bfe2-edb6a526c729 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:39:24.924936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5190289-81d9-4a16-bfe2-edb6a526c729 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:39:24.928520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef5b32-f4df-458a-bde9-dbc8d8d2c51a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:39:24.929156 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef5b32-f4df-458a-bde9-dbc8d8d2c51a service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:39:24.929440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef5b32-f4df-458a-bde9-dbc8d8d2c51a service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:39:24.929715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef5b32-f4df-458a-bde9-dbc8d8d2c51a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:39:24.938435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef5b32-f4df-458a-bde9-dbc8d8d2c51a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:39:24.939845 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef5b32-f4df-458a-bde9-dbc8d8d2c51a service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:39:24.949178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1330/2662] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:39:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:39:34.944559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=10 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:39:54.508498 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397245b-9182-48ae-a6b8-f4b5f0c9e86a service placement] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:39:54.510443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a397245b-9182-48ae-a6b8-f4b5f0c9e86a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:39:54.522340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397245b-9182-48ae-a6b8-f4b5f0c9e86a service placement] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:39:54.526595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa0c1f-ed39-4c3c-9900-5585dd3fa67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:39:54.527331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa0c1f-ed39-4c3c-9900-5585dd3fa67b service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:39:54.527610 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa0c1f-ed39-4c3c-9900-5585dd3fa67b service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:39:54.527883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa0c1f-ed39-4c3c-9900-5585dd3fa67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:39:54.529582 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0daa0c1f-ed39-4c3c-9900-5585dd3fa67b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:39:54.604767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa0c1f-ed39-4c3c-9900-5585dd3fa67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:39:54.606251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa0c1f-ed39-4c3c-9900-5585dd3fa67b service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:39:54.626210 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1333/2663] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:39:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:39:55.824279 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef5b32-f4df-458a-bde9-dbc8d8d2c51a service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:39:55.833474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef5b32-f4df-458a-bde9-dbc8d8d2c51a service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:39:55.837572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c4359-71fb-402f-aa43-2cc81e151aa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:39:55.838211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c4359-71fb-402f-aa43-2cc81e151aa1 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:39:55.838523 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c4359-71fb-402f-aa43-2cc81e151aa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:39:55.838794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c4359-71fb-402f-aa43-2cc81e151aa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:39:55.840842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8b9c4359-71fb-402f-aa43-2cc81e151aa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:39:55.855306 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c4359-71fb-402f-aa43-2cc81e151aa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:39:55.856746 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c4359-71fb-402f-aa43-2cc81e151aa1 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:39:55.868270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1331/2664] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:39:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:39:56.042314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa0c1f-ed39-4c3c-9900-5585dd3fa67b service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:39:56.044322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0daa0c1f-ed39-4c3c-9900-5585dd3fa67b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:39:56.063986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa0c1f-ed39-4c3c-9900-5585dd3fa67b service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:39:56.072294 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39698d5a-5f4d-4918-b817-cd5e211231e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:39:56.073227 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39698d5a-5f4d-4918-b817-cd5e211231e2 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:39:56.073521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39698d5a-5f4d-4918-b817-cd5e211231e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:39:56.073886 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39698d5a-5f4d-4918-b817-cd5e211231e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:39:56.075785 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-39698d5a-5f4d-4918-b817-cd5e211231e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:39:56.093311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39698d5a-5f4d-4918-b817-cd5e211231e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:39:56.095759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39698d5a-5f4d-4918-b817-cd5e211231e2 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:39:56.116669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1334/2665] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:39:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:39:58.875946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921637dc-c0c5-45ca-b5e9-04b29c655ece None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:39:58.876489 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921637dc-c0c5-45ca-b5e9-04b29c655ece None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:39:58.876965 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921637dc-c0c5-45ca-b5e9-04b29c655ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:39:58.905662 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-921637dc-c0c5-45ca-b5e9-04b29c655ece None None] MFA Rules not processed for user `e13a5203bc06439ba86befd2eb849ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:39:58.906890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-921637dc-c0c5-45ca-b5e9-04b29c655ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:39:58.913602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1332/2666] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:39:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:39:59.211203 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39698d5a-5f4d-4918-b817-cd5e211231e2 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:39:59.221483 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39698d5a-5f4d-4918-b817-cd5e211231e2 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:39:59.225082 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935eb98-919d-47ab-b9d1-b6ae5528945e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:39:59.225732 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935eb98-919d-47ab-b9d1-b6ae5528945e service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:39:59.226020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935eb98-919d-47ab-b9d1-b6ae5528945e service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:39:59.226336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935eb98-919d-47ab-b9d1-b6ae5528945e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:39:59.228194 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3935eb98-919d-47ab-b9d1-b6ae5528945e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:39:59.238803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935eb98-919d-47ab-b9d1-b6ae5528945e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:39:59.240261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935eb98-919d-47ab-b9d1-b6ae5528945e service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:39:59.248291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1335/2667] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:39:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:40:01.585797 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921637dc-c0c5-45ca-b5e9-04b29c655ece None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:01.587816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-921637dc-c0c5-45ca-b5e9-04b29c655ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:40:01.603368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921637dc-c0c5-45ca-b5e9-04b29c655ece None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:01.608400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d0c84-67dc-4959-85cb-53351618dc0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:01.609070 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d0c84-67dc-4959-85cb-53351618dc0e service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:01.609381 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d0c84-67dc-4959-85cb-53351618dc0e service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:01.609661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d0c84-67dc-4959-85cb-53351618dc0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:01.611418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-818d0c84-67dc-4959-85cb-53351618dc0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:40:01.622723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d0c84-67dc-4959-85cb-53351618dc0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:01.624376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d0c84-67dc-4959-85cb-53351618dc0e service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:01.634422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1333/2668] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:40:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:40:09.244689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=53 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:40:11.630188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=18 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:40:11.654623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935eb98-919d-47ab-b9d1-b6ae5528945e service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:11.668642 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935eb98-919d-47ab-b9d1-b6ae5528945e service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:11.674067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed9ef0-8d57-4c68-a6bb-07d6a5d3a950 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:11.674738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed9ef0-8d57-4c68-a6bb-07d6a5d3a950 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:11.675019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed9ef0-8d57-4c68-a6bb-07d6a5d3a950 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:11.675292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed9ef0-8d57-4c68-a6bb-07d6a5d3a950 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:11.677022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-10ed9ef0-8d57-4c68-a6bb-07d6a5d3a950 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:40:11.689539 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed9ef0-8d57-4c68-a6bb-07d6a5d3a950 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:11.691184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed9ef0-8d57-4c68-a6bb-07d6a5d3a950 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:11.702349 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1336/2669] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:40:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:40:15.482013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d0c84-67dc-4959-85cb-53351618dc0e service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:15.483566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-818d0c84-67dc-4959-85cb-53351618dc0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:40:15.499043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d0c84-67dc-4959-85cb-53351618dc0e service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:15.504683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd5bf5-9537-475a-a523-59978029bc50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:15.505333 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd5bf5-9537-475a-a523-59978029bc50 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:15.505649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd5bf5-9537-475a-a523-59978029bc50 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:15.505928 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd5bf5-9537-475a-a523-59978029bc50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:15.507717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8bcd5bf5-9537-475a-a523-59978029bc50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:40:15.519974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd5bf5-9537-475a-a523-59978029bc50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:15.521560 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd5bf5-9537-475a-a523-59978029bc50 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:15.532385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1334/2670] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:40:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:40:21.697872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=6 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:40:25.527669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:31.258681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed9ef0-8d57-4c68-a6bb-07d6a5d3a950 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:31.267956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed9ef0-8d57-4c68-a6bb-07d6a5d3a950 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:31.271463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511d547-5c97-4541-95bd-81fbdc708dcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:31.272093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511d547-5c97-4541-95bd-81fbdc708dcd service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:31.272375 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511d547-5c97-4541-95bd-81fbdc708dcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:31.272649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511d547-5c97-4541-95bd-81fbdc708dcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:31.280945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511d547-5c97-4541-95bd-81fbdc708dcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:31.282410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511d547-5c97-4541-95bd-81fbdc708dcd service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:31.309735 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1337/2671] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:33.750124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd5bf5-9537-475a-a523-59978029bc50 service nova] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:33.760900 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd5bf5-9537-475a-a523-59978029bc50 service nova] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:33.766887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a3466-8474-4dab-8ed8-9c76183a8d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:33.767562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3466-8474-4dab-8ed8-9c76183a8d23 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:33.767851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3466-8474-4dab-8ed8-9c76183a8d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:33.768763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3466-8474-4dab-8ed8-9c76183a8d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:33.770649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-168a3466-8474-4dab-8ed8-9c76183a8d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:33.841067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a3466-8474-4dab-8ed8-9c76183a8d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:33.842577 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a3466-8474-4dab-8ed8-9c76183a8d23 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:33.852857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1335/2672] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:37.295049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511d547-5c97-4541-95bd-81fbdc708dcd service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:37.309503 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511d547-5c97-4541-95bd-81fbdc708dcd service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:37.315692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6864d-09f7-43b2-95ce-cb04e6b2654c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:37.316679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6864d-09f7-43b2-95ce-cb04e6b2654c service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.317181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6864d-09f7-43b2-95ce-cb04e6b2654c service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.317868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6864d-09f7-43b2-95ce-cb04e6b2654c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.321311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-60b6864d-09f7-43b2-95ce-cb04e6b2654c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:37.342372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6864d-09f7-43b2-95ce-cb04e6b2654c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:37.344345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6864d-09f7-43b2-95ce-cb04e6b2654c service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:37.357523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1338/2673] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:37.365940 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a3466-8474-4dab-8ed8-9c76183a8d23 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:37.367355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-168a3466-8474-4dab-8ed8-9c76183a8d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:37.384168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a3466-8474-4dab-8ed8-9c76183a8d23 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:37.391721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c36fd-a86e-46a3-bfd5-cbc8dad96cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:37.392709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c36fd-a86e-46a3-bfd5-cbc8dad96cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.393040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c36fd-a86e-46a3-bfd5-cbc8dad96cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.393384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c36fd-a86e-46a3-bfd5-cbc8dad96cff admin admin] PATH_INFO: `/v3/users/cb3d46a7a0ae40e9b157b7199b18de27` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.394801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c36fd-a86e-46a3-bfd5-cbc8dad96cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3d46a7a0ae40e9b157b7199b18de27)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:37.398196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c36fd-a86e-46a3-bfd5-cbc8dad96cff admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:37.452062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-518c36fd-a86e-46a3-bfd5-cbc8dad96cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3d46a7a0ae40e9b157b7199b18de27'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:37.465294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-518c36fd-a86e-46a3-bfd5-cbc8dad96cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3d46a7a0ae40e9b157b7199b18de27'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:37.474563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-518c36fd-a86e-46a3-bfd5-cbc8dad96cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3d46a7a0ae40e9b157b7199b18de27'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:37.475178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-518c36fd-a86e-46a3-bfd5-cbc8dad96cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3d46a7a0ae40e9b157b7199b18de27'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:37.482458 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-518c36fd-a86e-46a3-bfd5-cbc8dad96cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3d46a7a0ae40e9b157b7199b18de27'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:37.488450 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1336/2674] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:41:37 2022] DELETE /identity/v3/users/cb3d46a7a0ae40e9b157b7199b18de27 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:37.665221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6864d-09f7-43b2-95ce-cb04e6b2654c service placement] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:37.676576 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6864d-09f7-43b2-95ce-cb04e6b2654c service placement] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:37.734702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:37.735567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.735854 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.736128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] PATH_INFO: `/v3/projects/728de65d013a489594d837589c7b978b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.737309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=728de65d013a489594d837589c7b978b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:37.739280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:37.789936 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728de65d013a489594d837589c7b978b', 'request_id': 'req-b55762a1-f525-43fc-9285-d71526e36aee'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:37.799323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728de65d013a489594d837589c7b978b', 'request_id': 'req-b55762a1-f525-43fc-9285-d71526e36aee'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:37.810154 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728de65d013a489594d837589c7b978b', 'request_id': 'req-b55762a1-f525-43fc-9285-d71526e36aee'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:37.823794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] The token cache is being invalidate because project 728de65d013a489594d837589c7b978b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:41:37.824081 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:37.824748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b55762a1-f525-43fc-9285-d71526e36aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:37.826347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1339/2675] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:41:37 2022] DELETE /identity/v3/projects/728de65d013a489594d837589c7b978b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:37.831295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85868eed-956b-47f9-9a1d-634c38a94049 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.831595 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85868eed-956b-47f9-9a1d-634c38a94049 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.831875 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85868eed-956b-47f9-9a1d-634c38a94049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.891836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-85868eed-956b-47f9-9a1d-634c38a94049 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:37.893057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-85868eed-956b-47f9-9a1d-634c38a94049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:37.923787 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:37.963482 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1337/2676] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:37.965207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc9088-53dd-44a6-9cdd-161fe7e5a37c None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.965501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc9088-53dd-44a6-9cdd-161fe7e5a37c None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.965782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc9088-53dd-44a6-9cdd-161fe7e5a37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.985145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-aadc9088-53dd-44a6-9cdd-161fe7e5a37c None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:37.986405 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-aadc9088-53dd-44a6-9cdd-161fe7e5a37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:37.997451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1340/2677] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:37.998714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85868eed-956b-47f9-9a1d-634c38a94049 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.011555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85868eed-956b-47f9-9a1d-634c38a94049 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.014454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5e067-89db-490e-b2e1-1a704aa6a392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.015152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5e067-89db-490e-b2e1-1a704aa6a392 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.015440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5e067-89db-490e-b2e1-1a704aa6a392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.015718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5e067-89db-490e-b2e1-1a704aa6a392 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.016454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5e067-89db-490e-b2e1-1a704aa6a392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:38.016773 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5e067-89db-490e-b2e1-1a704aa6a392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.018411 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5e067-89db-490e-b2e1-1a704aa6a392 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.031054 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1338/2678] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:38.035141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc9088-53dd-44a6-9cdd-161fe7e5a37c None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.044679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc9088-53dd-44a6-9cdd-161fe7e5a37c None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.049928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68176ea-97e1-4f71-9d63-dda2a58e5044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.050834 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68176ea-97e1-4f71-9d63-dda2a58e5044 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.051019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68176ea-97e1-4f71-9d63-dda2a58e5044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.051291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68176ea-97e1-4f71-9d63-dda2a58e5044 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.051982 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68176ea-97e1-4f71-9d63-dda2a58e5044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.053908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68176ea-97e1-4f71-9d63-dda2a58e5044 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.093123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1341/2679] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:41:38 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:38.097514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5e067-89db-490e-b2e1-1a704aa6a392 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.107887 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5e067-89db-490e-b2e1-1a704aa6a392 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.169463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759cf50-b1ae-487a-be4a-0e6463553ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.170129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759cf50-b1ae-487a-be4a-0e6463553ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.170519 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759cf50-b1ae-487a-be4a-0e6463553ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.170846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759cf50-b1ae-487a-be4a-0e6463553ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.171613 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759cf50-b1ae-487a-be4a-0e6463553ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.173781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759cf50-b1ae-487a-be4a-0e6463553ae1 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.195427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1339/2680] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:41:38 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:38.199603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68176ea-97e1-4f71-9d63-dda2a58e5044 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.210623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68176ea-97e1-4f71-9d63-dda2a58e5044 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.215770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0277a81-5d5c-4ff8-9d55-b8d785ade08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.216443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0277a81-5d5c-4ff8-9d55-b8d785ade08d admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.216737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0277a81-5d5c-4ff8-9d55-b8d785ade08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.217016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0277a81-5d5c-4ff8-9d55-b8d785ade08d admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.217814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0277a81-5d5c-4ff8-9d55-b8d785ade08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.219470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0277a81-5d5c-4ff8-9d55-b8d785ade08d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.225251 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:38.230995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1342/2681] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:41:38.234777 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759cf50-b1ae-487a-be4a-0e6463553ae1 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.244776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759cf50-b1ae-487a-be4a-0e6463553ae1 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.249653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85b59d-ce38-4cbb-b60f-f52200ab7ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.250409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85b59d-ce38-4cbb-b60f-f52200ab7ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.250677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85b59d-ce38-4cbb-b60f-f52200ab7ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.251011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85b59d-ce38-4cbb-b60f-f52200ab7ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.251743 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85b59d-ce38-4cbb-b60f-f52200ab7ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.253599 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85b59d-ce38-4cbb-b60f-f52200ab7ef4 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.267124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1340/2682] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:41:38.268393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0277a81-5d5c-4ff8-9d55-b8d785ade08d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.279489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0277a81-5d5c-4ff8-9d55-b8d785ade08d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.284392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fbb09-520c-486e-ae3c-ddfceee698aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.285297 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fbb09-520c-486e-ae3c-ddfceee698aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.285577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fbb09-520c-486e-ae3c-ddfceee698aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.285851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fbb09-520c-486e-ae3c-ddfceee698aa admin admin] PATH_INFO: `/v3/projects/2d6a4d54fbd6409290cae6fed90d1a46/users/7d68a865a4ff4befb17fa64bf2a87a66/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.300336 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fbb09-520c-486e-ae3c-ddfceee698aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6a4d54fbd6409290cae6fed90d1a46, user_id=7d68a865a4ff4befb17fa64bf2a87a66, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.303528 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fbb09-520c-486e-ae3c-ddfceee698aa admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.316131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1343/2683] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:38 2022] PUT /identity/v3/projects/2d6a4d54fbd6409290cae6fed90d1a46/users/7d68a865a4ff4befb17fa64bf2a87a66/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:38.327136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daf731-f3f8-4c49-a926-99d6706f9841 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.327426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daf731-f3f8-4c49-a926-99d6706f9841 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.327702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daf731-f3f8-4c49-a926-99d6706f9841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.354243 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-72daf731-f3f8-4c49-a926-99d6706f9841 None None] MFA Rules not processed for user `7d68a865a4ff4befb17fa64bf2a87a66`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:38.355460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-72daf731-f3f8-4c49-a926-99d6706f9841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:38.418792 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1341/2684] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:38.423188 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fbb09-520c-486e-ae3c-ddfceee698aa admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.483106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fbb09-520c-486e-ae3c-ddfceee698aa admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.487689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870ba0d-2be5-46fc-a83b-906711926c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.488332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870ba0d-2be5-46fc-a83b-906711926c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.488611 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870ba0d-2be5-46fc-a83b-906711926c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.488883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870ba0d-2be5-46fc-a83b-906711926c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.489555 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870ba0d-2be5-46fc-a83b-906711926c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.492307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870ba0d-2be5-46fc-a83b-906711926c0d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.516898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1344/2685] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:41:38 2022] POST /identity/v3/projects => generated 405 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:38.520810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daf731-f3f8-4c49-a926-99d6706f9841 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.532741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daf731-f3f8-4c49-a926-99d6706f9841 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.561734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:38.590098 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bd2da-a977-49cf-84b9-f18f9aa347de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.591486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bd2da-a977-49cf-84b9-f18f9aa347de admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.591778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bd2da-a977-49cf-84b9-f18f9aa347de admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.592058 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bd2da-a977-49cf-84b9-f18f9aa347de admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.592752 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bd2da-a977-49cf-84b9-f18f9aa347de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.594738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bd2da-a977-49cf-84b9-f18f9aa347de admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.613964 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1342/2686] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:41:38 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:38.618643 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870ba0d-2be5-46fc-a83b-906711926c0d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.628887 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870ba0d-2be5-46fc-a83b-906711926c0d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.634125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c649f6-b8b0-404d-a3f9-94ec8782fa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.634818 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c649f6-b8b0-404d-a3f9-94ec8782fa1d admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.635105 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c649f6-b8b0-404d-a3f9-94ec8782fa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.635385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c649f6-b8b0-404d-a3f9-94ec8782fa1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.636109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c649f6-b8b0-404d-a3f9-94ec8782fa1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.638158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c649f6-b8b0-404d-a3f9-94ec8782fa1d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.648867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1345/2687] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:41:38.652875 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bd2da-a977-49cf-84b9-f18f9aa347de admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.663157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bd2da-a977-49cf-84b9-f18f9aa347de admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.667627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2049e89-c8ca-469f-b1e8-709c539ce276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.668491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2049e89-c8ca-469f-b1e8-709c539ce276 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.668775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2049e89-c8ca-469f-b1e8-709c539ce276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.669045 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2049e89-c8ca-469f-b1e8-709c539ce276 admin admin] PATH_INFO: `/v3/projects/db563033dbcb4a0c872fe18dcf1535bc/users/0de555dc732e455e94fcaed045f44e23/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.683832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2049e89-c8ca-469f-b1e8-709c539ce276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db563033dbcb4a0c872fe18dcf1535bc, user_id=0de555dc732e455e94fcaed045f44e23, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.686489 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2049e89-c8ca-469f-b1e8-709c539ce276 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.698883 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1343/2688] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:38 2022] PUT /identity/v3/projects/db563033dbcb4a0c872fe18dcf1535bc/users/0de555dc732e455e94fcaed045f44e23/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:38.702678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c649f6-b8b0-404d-a3f9-94ec8782fa1d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.712825 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c649f6-b8b0-404d-a3f9-94ec8782fa1d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.767932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf01bd-d70d-4b4b-b38d-34f2b97e60be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.768649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf01bd-d70d-4b4b-b38d-34f2b97e60be admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.768930 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf01bd-d70d-4b4b-b38d-34f2b97e60be admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.769203 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf01bd-d70d-4b4b-b38d-34f2b97e60be admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.769954 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf01bd-d70d-4b4b-b38d-34f2b97e60be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.771689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf01bd-d70d-4b4b-b38d-34f2b97e60be admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.781542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1346/2689] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:41:38.785543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2049e89-c8ca-469f-b1e8-709c539ce276 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.800669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2049e89-c8ca-469f-b1e8-709c539ce276 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.815072 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce48d42-5d65-42e6-8f8f-a2e9df5168c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.815902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48d42-5d65-42e6-8f8f-a2e9df5168c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.816179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48d42-5d65-42e6-8f8f-a2e9df5168c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.816450 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48d42-5d65-42e6-8f8f-a2e9df5168c1 admin admin] PATH_INFO: `/v3/projects/db563033dbcb4a0c872fe18dcf1535bc/users/0de555dc732e455e94fcaed045f44e23/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.818485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce48d42-5d65-42e6-8f8f-a2e9df5168c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db563033dbcb4a0c872fe18dcf1535bc, user_id=0de555dc732e455e94fcaed045f44e23, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.821163 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce48d42-5d65-42e6-8f8f-a2e9df5168c1 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.831849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1344/2690] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:38 2022] PUT /identity/v3/projects/db563033dbcb4a0c872fe18dcf1535bc/users/0de555dc732e455e94fcaed045f44e23/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:38.843356 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ce65b-be53-4e69-9fc8-ca9b86d6f7fc None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.843654 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ce65b-be53-4e69-9fc8-ca9b86d6f7fc None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.843989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ce65b-be53-4e69-9fc8-ca9b86d6f7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.870874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-174ce65b-be53-4e69-9fc8-ca9b86d6f7fc None None] MFA Rules not processed for user `0de555dc732e455e94fcaed045f44e23`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:38.872083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-174ce65b-be53-4e69-9fc8-ca9b86d6f7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:38.905045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:38.940551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1347/2691] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:38 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:38.951149 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce48d42-5d65-42e6-8f8f-a2e9df5168c1 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.952873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9ce48d42-5d65-42e6-8f8f-a2e9df5168c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:39.008837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce48d42-5d65-42e6-8f8f-a2e9df5168c1 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:39.012504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073436c-5ed5-47e3-b35d-2f28ede8ce6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:39.013146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073436c-5ed5-47e3-b35d-2f28ede8ce6f service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.013435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073436c-5ed5-47e3-b35d-2f28ede8ce6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.013718 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073436c-5ed5-47e3-b35d-2f28ede8ce6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.022591 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073436c-5ed5-47e3-b35d-2f28ede8ce6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:39.024020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073436c-5ed5-47e3-b35d-2f28ede8ce6f service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:39.033333 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1345/2692] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:38 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:39.164110 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ce65b-be53-4e69-9fc8-ca9b86d6f7fc None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:39.230952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ce65b-be53-4e69-9fc8-ca9b86d6f7fc None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:39.235408 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073436c-5ed5-47e3-b35d-2f28ede8ce6f service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:39.235638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e0cf8-8907-4141-905e-3bd9c9ef4083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:39.236521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e0cf8-8907-4141-905e-3bd9c9ef4083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.236801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e0cf8-8907-4141-905e-3bd9c9ef4083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.236897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c073436c-5ed5-47e3-b35d-2f28ede8ce6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:39.237079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e0cf8-8907-4141-905e-3bd9c9ef4083 admin admin] PATH_INFO: `/v3/users/7d68a865a4ff4befb17fa64bf2a87a66` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.238264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e0cf8-8907-4141-905e-3bd9c9ef4083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d68a865a4ff4befb17fa64bf2a87a66)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:39.240240 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e0cf8-8907-4141-905e-3bd9c9ef4083 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:39.289600 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-494e0cf8-8907-4141-905e-3bd9c9ef4083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d68a865a4ff4befb17fa64bf2a87a66'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:39.297062 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-494e0cf8-8907-4141-905e-3bd9c9ef4083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d68a865a4ff4befb17fa64bf2a87a66'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:39.297211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073436c-5ed5-47e3-b35d-2f28ede8ce6f service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:39.297838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-494e0cf8-8907-4141-905e-3bd9c9ef4083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d68a865a4ff4befb17fa64bf2a87a66'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:39.298544 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-494e0cf8-8907-4141-905e-3bd9c9ef4083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d68a865a4ff4befb17fa64bf2a87a66'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:39.301186 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25054635-d035-46fc-82db-0e2b46b54222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:39.301838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25054635-d035-46fc-82db-0e2b46b54222 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.302121 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25054635-d035-46fc-82db-0e2b46b54222 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.302442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25054635-d035-46fc-82db-0e2b46b54222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.304263 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-25054635-d035-46fc-82db-0e2b46b54222 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:39.311170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-494e0cf8-8907-4141-905e-3bd9c9ef4083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d68a865a4ff4befb17fa64bf2a87a66'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:39.316301 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25054635-d035-46fc-82db-0e2b46b54222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:39.317815 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25054635-d035-46fc-82db-0e2b46b54222 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:39.323262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1348/2693] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:41:39 2022] DELETE /identity/v3/users/7d68a865a4ff4befb17fa64bf2a87a66 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:39.327614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d468f-eefa-4293-a993-2344ad5e1807 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.327905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d468f-eefa-4293-a993-2344ad5e1807 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.328006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1346/2694] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:41:39.328186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d468f-eefa-4293-a993-2344ad5e1807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.346976 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a95d468f-eefa-4293-a993-2344ad5e1807 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:39.348166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a95d468f-eefa-4293-a993-2344ad5e1807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:39.395569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:39.412693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1349/2695] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:41:39.422339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25054635-d035-46fc-82db-0e2b46b54222 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:39.423799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-25054635-d035-46fc-82db-0e2b46b54222 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:39.434979 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25054635-d035-46fc-82db-0e2b46b54222 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:39.465658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:39.490465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa59d5b-8a43-4b64-97a8-6d0bdc6a9acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:39.491145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa59d5b-8a43-4b64-97a8-6d0bdc6a9acd service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.491429 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa59d5b-8a43-4b64-97a8-6d0bdc6a9acd service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.491806 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa59d5b-8a43-4b64-97a8-6d0bdc6a9acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.501041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa59d5b-8a43-4b64-97a8-6d0bdc6a9acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:39.502503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa59d5b-8a43-4b64-97a8-6d0bdc6a9acd service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:39.512616 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1347/2696] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:39.787553 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d468f-eefa-4293-a993-2344ad5e1807 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:39.789080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a95d468f-eefa-4293-a993-2344ad5e1807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:39.801317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d468f-eefa-4293-a993-2344ad5e1807 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:39.808506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:39.809380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.809658 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.809928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] PATH_INFO: `/v3/projects/2d6a4d54fbd6409290cae6fed90d1a46` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.811148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6a4d54fbd6409290cae6fed90d1a46)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:39.813071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:39.864085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6a4d54fbd6409290cae6fed90d1a46', 'request_id': 'req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:39.872880 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6a4d54fbd6409290cae6fed90d1a46', 'request_id': 'req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:39.879425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6a4d54fbd6409290cae6fed90d1a46', 'request_id': 'req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:39.891191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] The token cache is being invalidate because project 2d6a4d54fbd6409290cae6fed90d1a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:41:39.891482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:39.892151 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:39.893701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1350/2697] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:41:39 2022] DELETE /identity/v3/projects/2d6a4d54fbd6409290cae6fed90d1a46 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:39.897658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa59d5b-8a43-4b64-97a8-6d0bdc6a9acd service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:39.907150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa59d5b-8a43-4b64-97a8-6d0bdc6a9acd service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:39.963867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c5e8c-61df-4d06-9acb-4d5c09baeba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:39.964742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c5e8c-61df-4d06-9acb-4d5c09baeba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.965028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c5e8c-61df-4d06-9acb-4d5c09baeba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.965320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c5e8c-61df-4d06-9acb-4d5c09baeba4 admin admin] PATH_INFO: `/v3/users/0de555dc732e455e94fcaed045f44e23` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.966499 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c5e8c-61df-4d06-9acb-4d5c09baeba4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0de555dc732e455e94fcaed045f44e23)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:39.968407 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c5e8c-61df-4d06-9acb-4d5c09baeba4 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:40.015101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-301c5e8c-61df-4d06-9acb-4d5c09baeba4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0de555dc732e455e94fcaed045f44e23'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:40.027757 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-301c5e8c-61df-4d06-9acb-4d5c09baeba4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0de555dc732e455e94fcaed045f44e23'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:40.035255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-301c5e8c-61df-4d06-9acb-4d5c09baeba4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0de555dc732e455e94fcaed045f44e23'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:40.035989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-301c5e8c-61df-4d06-9acb-4d5c09baeba4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0de555dc732e455e94fcaed045f44e23'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:40.042990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-301c5e8c-61df-4d06-9acb-4d5c09baeba4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0de555dc732e455e94fcaed045f44e23'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:40.050712 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1348/2698] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:41:39 2022] DELETE /identity/v3/users/0de555dc732e455e94fcaed045f44e23 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:40.346678 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.351035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:40.369074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe960d-be61-4c0c-bd45-5c6cc5734aaa admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.445960 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.446871 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.447148 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.447435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] PATH_INFO: `/v3/projects/db563033dbcb4a0c872fe18dcf1535bc` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.448928 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=db563033dbcb4a0c872fe18dcf1535bc)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:40.451462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:40.498395 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db563033dbcb4a0c872fe18dcf1535bc', 'request_id': 'req-c9243e79-fbea-4476-a6df-689fd6cf2e4d'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:40.506793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db563033dbcb4a0c872fe18dcf1535bc', 'request_id': 'req-c9243e79-fbea-4476-a6df-689fd6cf2e4d'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:40.512456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db563033dbcb4a0c872fe18dcf1535bc', 'request_id': 'req-c9243e79-fbea-4476-a6df-689fd6cf2e4d'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:40.524086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] The token cache is being invalidate because project db563033dbcb4a0c872fe18dcf1535bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:41:40.524370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:40.525035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c9243e79-fbea-4476-a6df-689fd6cf2e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:40.526941 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1351/2699] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:41:40 2022] DELETE /identity/v3/projects/db563033dbcb4a0c872fe18dcf1535bc => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:40.531615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf630-47b1-447c-83ba-2ff677195b78 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.532075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf630-47b1-447c-83ba-2ff677195b78 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.532223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf630-47b1-447c-83ba-2ff677195b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.549554 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7d2cf630-47b1-447c-83ba-2ff677195b78 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:40.550821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7d2cf630-47b1-447c-83ba-2ff677195b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:40.597574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:40.612985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1349/2700] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:40.621134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff208e9c-e712-4382-84fd-9c2b406b95d4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.622114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff208e9c-e712-4382-84fd-9c2b406b95d4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.622491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff208e9c-e712-4382-84fd-9c2b406b95d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.630564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:40.642793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ff208e9c-e712-4382-84fd-9c2b406b95d4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:40.644681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ff208e9c-e712-4382-84fd-9c2b406b95d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:40.652647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1352/2701] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:40.657050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cf630-47b1-447c-83ba-2ff677195b78 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.667245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cf630-47b1-447c-83ba-2ff677195b78 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.673772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9f749-c0d2-42b8-84e4-3149805ac832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.673772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9f749-c0d2-42b8-84e4-3149805ac832 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.673772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9f749-c0d2-42b8-84e4-3149805ac832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.673772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9f749-c0d2-42b8-84e4-3149805ac832 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.674988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9f749-c0d2-42b8-84e4-3149805ac832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:40.675330 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9f749-c0d2-42b8-84e4-3149805ac832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:40.676957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9f749-c0d2-42b8-84e4-3149805ac832 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:40.694187 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1350/2702] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:40.695229 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff208e9c-e712-4382-84fd-9c2b406b95d4 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.705126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff208e9c-e712-4382-84fd-9c2b406b95d4 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.710133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577bb3e-6250-45cc-83b8-355088a014e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.710873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577bb3e-6250-45cc-83b8-355088a014e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.711169 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577bb3e-6250-45cc-83b8-355088a014e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.711521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577bb3e-6250-45cc-83b8-355088a014e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.712598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577bb3e-6250-45cc-83b8-355088a014e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:40.714702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577bb3e-6250-45cc-83b8-355088a014e3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:40.750943 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1353/2703] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:41:40 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:40.752226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9f749-c0d2-42b8-84e4-3149805ac832 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.764647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9f749-c0d2-42b8-84e4-3149805ac832 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.840889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c4e06-ae03-4fab-be5e-b7d16fc160b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.841678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c4e06-ae03-4fab-be5e-b7d16fc160b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.842073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c4e06-ae03-4fab-be5e-b7d16fc160b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.842481 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c4e06-ae03-4fab-be5e-b7d16fc160b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.843276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c4e06-ae03-4fab-be5e-b7d16fc160b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:40.845386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c4e06-ae03-4fab-be5e-b7d16fc160b1 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:40.867843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1351/2704] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:41:40 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:40.871906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577bb3e-6250-45cc-83b8-355088a014e3 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.886474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577bb3e-6250-45cc-83b8-355088a014e3 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.891628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f320a4-d09b-494e-857a-ef4534074d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.892370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f320a4-d09b-494e-857a-ef4534074d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.892613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f320a4-d09b-494e-857a-ef4534074d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.892895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f320a4-d09b-494e-857a-ef4534074d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.893645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f320a4-d09b-494e-857a-ef4534074d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:40.895492 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f320a4-d09b-494e-857a-ef4534074d44 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:40.911608 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1354/2705] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:41:40.913154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c4e06-ae03-4fab-be5e-b7d16fc160b1 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.925377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c4e06-ae03-4fab-be5e-b7d16fc160b1 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.931418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbf10b-8a52-42c8-a5ea-dabb930fe952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.932119 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbf10b-8a52-42c8-a5ea-dabb930fe952 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.932410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbf10b-8a52-42c8-a5ea-dabb930fe952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.932699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbf10b-8a52-42c8-a5ea-dabb930fe952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.933441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbf10b-8a52-42c8-a5ea-dabb930fe952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:40.935117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbf10b-8a52-42c8-a5ea-dabb930fe952 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:40.947567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1352/2706] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:41:40.951596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f320a4-d09b-494e-857a-ef4534074d44 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.962796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f320a4-d09b-494e-857a-ef4534074d44 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.968388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea0a29-619d-49cb-a6df-a0495830fbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.969319 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea0a29-619d-49cb-a6df-a0495830fbe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.969604 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea0a29-619d-49cb-a6df-a0495830fbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.969882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea0a29-619d-49cb-a6df-a0495830fbe7 admin admin] PATH_INFO: `/v3/projects/8486d3db33e445c0ba4b72ea802eb1fe/users/01c4ed0a470f46a3b04113ee7649cb29/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.986964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea0a29-619d-49cb-a6df-a0495830fbe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8486d3db33e445c0ba4b72ea802eb1fe, user_id=01c4ed0a470f46a3b04113ee7649cb29, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:40.989652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea0a29-619d-49cb-a6df-a0495830fbe7 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.005069 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1355/2707] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:40 2022] PUT /identity/v3/projects/8486d3db33e445c0ba4b72ea802eb1fe/users/01c4ed0a470f46a3b04113ee7649cb29/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:41.017034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df795b21-1689-45fb-a2b1-ab2c32c5ba9a None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.017034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df795b21-1689-45fb-a2b1-ab2c32c5ba9a None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.017034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df795b21-1689-45fb-a2b1-ab2c32c5ba9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.046167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-df795b21-1689-45fb-a2b1-ab2c32c5ba9a None None] MFA Rules not processed for user `01c4ed0a470f46a3b04113ee7649cb29`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:41.047444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-df795b21-1689-45fb-a2b1-ab2c32c5ba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:41.122668 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1353/2708] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:41.123776 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea0a29-619d-49cb-a6df-a0495830fbe7 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.194170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea0a29-619d-49cb-a6df-a0495830fbe7 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.199605 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd2dde-fcce-4a51-8249-af6c5d6986d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.200287 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd2dde-fcce-4a51-8249-af6c5d6986d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.200593 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd2dde-fcce-4a51-8249-af6c5d6986d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.200932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd2dde-fcce-4a51-8249-af6c5d6986d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.201645 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd2dde-fcce-4a51-8249-af6c5d6986d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.203630 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd2dde-fcce-4a51-8249-af6c5d6986d1 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.232020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1356/2709] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:41:41 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:41.235933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df795b21-1689-45fb-a2b1-ab2c32c5ba9a None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.250755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df795b21-1689-45fb-a2b1-ab2c32c5ba9a None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.309936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:41.327586 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3c8dd-529d-4f72-8275-66791b1e58eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.328287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3c8dd-529d-4f72-8275-66791b1e58eb admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.328572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3c8dd-529d-4f72-8275-66791b1e58eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.328851 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3c8dd-529d-4f72-8275-66791b1e58eb admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.331191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3c8dd-529d-4f72-8275-66791b1e58eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.334012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3c8dd-529d-4f72-8275-66791b1e58eb admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.355136 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1354/2710] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:41:41 2022] POST /identity/v3/users => generated 295 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:41.359075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd2dde-fcce-4a51-8249-af6c5d6986d1 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.369989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd2dde-fcce-4a51-8249-af6c5d6986d1 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.374607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5113c-0060-4c2c-8861-d3913dbae3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.375257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5113c-0060-4c2c-8861-d3913dbae3ee admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.375540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5113c-0060-4c2c-8861-d3913dbae3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.375821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5113c-0060-4c2c-8861-d3913dbae3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.376540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5113c-0060-4c2c-8861-d3913dbae3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.378157 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5113c-0060-4c2c-8861-d3913dbae3ee admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.391893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1357/2711] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:41:41.395827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3c8dd-529d-4f72-8275-66791b1e58eb admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.407050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3c8dd-529d-4f72-8275-66791b1e58eb admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.411769 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840d6d4-9608-4529-a306-d56a7be4b989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.412708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840d6d4-9608-4529-a306-d56a7be4b989 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.413073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840d6d4-9608-4529-a306-d56a7be4b989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.413255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840d6d4-9608-4529-a306-d56a7be4b989 admin admin] PATH_INFO: `/v3/projects/71f3c77fa9b44737ae0b04f0523e384f/users/15c3b305651d4e85a08fc242b101ad6f/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.429727 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840d6d4-9608-4529-a306-d56a7be4b989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f3c77fa9b44737ae0b04f0523e384f, user_id=15c3b305651d4e85a08fc242b101ad6f, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.433094 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840d6d4-9608-4529-a306-d56a7be4b989 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.447562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1355/2712] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:41 2022] PUT /identity/v3/projects/71f3c77fa9b44737ae0b04f0523e384f/users/15c3b305651d4e85a08fc242b101ad6f/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:41.451570 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5113c-0060-4c2c-8861-d3913dbae3ee admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.462093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5113c-0060-4c2c-8861-d3913dbae3ee admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.521344 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf4a53-e1ad-437d-96e3-9c95e9c0efcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.522019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf4a53-e1ad-437d-96e3-9c95e9c0efcd admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.522332 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf4a53-e1ad-437d-96e3-9c95e9c0efcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.522619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf4a53-e1ad-437d-96e3-9c95e9c0efcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.523357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf4a53-e1ad-437d-96e3-9c95e9c0efcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.524971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf4a53-e1ad-437d-96e3-9c95e9c0efcd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.534973 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1358/2713] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:41:41.544516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840d6d4-9608-4529-a306-d56a7be4b989 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.555738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840d6d4-9608-4529-a306-d56a7be4b989 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.560330 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df289a4-a588-44dd-958f-1bedb6d041cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.561215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df289a4-a588-44dd-958f-1bedb6d041cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.561497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df289a4-a588-44dd-958f-1bedb6d041cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.561775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df289a4-a588-44dd-958f-1bedb6d041cc admin admin] PATH_INFO: `/v3/projects/71f3c77fa9b44737ae0b04f0523e384f/users/15c3b305651d4e85a08fc242b101ad6f/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.563758 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df289a4-a588-44dd-958f-1bedb6d041cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f3c77fa9b44737ae0b04f0523e384f, user_id=15c3b305651d4e85a08fc242b101ad6f, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.566418 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df289a4-a588-44dd-958f-1bedb6d041cc admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.577774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1356/2714] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:41 2022] PUT /identity/v3/projects/71f3c77fa9b44737ae0b04f0523e384f/users/15c3b305651d4e85a08fc242b101ad6f/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:41.588384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63264780-bcc6-422a-8130-3955bb29e0c7 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.588624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63264780-bcc6-422a-8130-3955bb29e0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.588910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63264780-bcc6-422a-8130-3955bb29e0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.607728 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:41.617524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-63264780-bcc6-422a-8130-3955bb29e0c7 None None] MFA Rules not processed for user `15c3b305651d4e85a08fc242b101ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:41.619105 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-63264780-bcc6-422a-8130-3955bb29e0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:41.704385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1359/2715] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:41.719803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df289a4-a588-44dd-958f-1bedb6d041cc admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.721385 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2df289a4-a588-44dd-958f-1bedb6d041cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:41.786197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df289a4-a588-44dd-958f-1bedb6d041cc admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.790410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2038d-f655-4623-865d-130417769e08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.791289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2038d-f655-4623-865d-130417769e08 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.791404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2038d-f655-4623-865d-130417769e08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.791607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2038d-f655-4623-865d-130417769e08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.801353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2038d-f655-4623-865d-130417769e08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.803673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2038d-f655-4623-865d-130417769e08 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.814898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1357/2716] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:41.913971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63264780-bcc6-422a-8130-3955bb29e0c7 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.924278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63264780-bcc6-422a-8130-3955bb29e0c7 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.928416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41d8fd-30d2-439f-b771-b7a9e9dc8980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.929135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41d8fd-30d2-439f-b771-b7a9e9dc8980 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.929469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41d8fd-30d2-439f-b771-b7a9e9dc8980 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.929882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41d8fd-30d2-439f-b771-b7a9e9dc8980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.992170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41d8fd-30d2-439f-b771-b7a9e9dc8980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.993569 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41d8fd-30d2-439f-b771-b7a9e9dc8980 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:42.003176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1360/2717] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:42.538859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2038d-f655-4623-865d-130417769e08 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:42.540330 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-30c2038d-f655-4623-865d-130417769e08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:42.603899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41d8fd-30d2-439f-b771-b7a9e9dc8980 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:42.605374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9d41d8fd-30d2-439f-b771-b7a9e9dc8980 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:42.619524 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2038d-f655-4623-865d-130417769e08 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:42.624669 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716acc4-9bbb-4744-97b7-42e2c5f4759b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:42.625563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716acc4-9bbb-4744-97b7-42e2c5f4759b service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:42.625671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716acc4-9bbb-4744-97b7-42e2c5f4759b service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:42.625894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716acc4-9bbb-4744-97b7-42e2c5f4759b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:42.627696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4716acc4-9bbb-4744-97b7-42e2c5f4759b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:42.640370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716acc4-9bbb-4744-97b7-42e2c5f4759b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:42.641889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716acc4-9bbb-4744-97b7-42e2c5f4759b service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:42.653464 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1358/2718] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:42.679995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41d8fd-30d2-439f-b771-b7a9e9dc8980 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:42.684647 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1722ddb-5154-4e33-9024-93b9917b3970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:42.685521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1722ddb-5154-4e33-9024-93b9917b3970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:42.685805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1722ddb-5154-4e33-9024-93b9917b3970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:42.686077 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1722ddb-5154-4e33-9024-93b9917b3970 admin admin] PATH_INFO: `/v3/users/e13a5203bc06439ba86befd2eb849ba5` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:42.687289 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1722ddb-5154-4e33-9024-93b9917b3970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13a5203bc06439ba86befd2eb849ba5)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:42.689221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1722ddb-5154-4e33-9024-93b9917b3970 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:42.727218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:42.736597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c1722ddb-5154-4e33-9024-93b9917b3970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13a5203bc06439ba86befd2eb849ba5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:42.744049 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c1722ddb-5154-4e33-9024-93b9917b3970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13a5203bc06439ba86befd2eb849ba5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:42.744923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c1722ddb-5154-4e33-9024-93b9917b3970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13a5203bc06439ba86befd2eb849ba5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:42.745615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c1722ddb-5154-4e33-9024-93b9917b3970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13a5203bc06439ba86befd2eb849ba5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:42.757927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-c1722ddb-5154-4e33-9024-93b9917b3970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13a5203bc06439ba86befd2eb849ba5'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:42.769770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1361/2719] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:41:42 2022] DELETE /identity/v3/users/e13a5203bc06439ba86befd2eb849ba5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:42.947663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716acc4-9bbb-4744-97b7-42e2c5f4759b service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:42.958360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716acc4-9bbb-4744-97b7-42e2c5f4759b service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:42.991312 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:43.019142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:43.019936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.020215 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.020487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] PATH_INFO: `/v3/projects/cb54f85be3994a40a4b52e576b36c6da` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.021670 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb54f85be3994a40a4b52e576b36c6da)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:43.023593 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:43.071448 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb54f85be3994a40a4b52e576b36c6da', 'request_id': 'req-324c1c39-541d-4bc3-8229-4d631e36f6e2'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:43.078755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb54f85be3994a40a4b52e576b36c6da', 'request_id': 'req-324c1c39-541d-4bc3-8229-4d631e36f6e2'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:43.084409 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb54f85be3994a40a4b52e576b36c6da', 'request_id': 'req-324c1c39-541d-4bc3-8229-4d631e36f6e2'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:43.095495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] The token cache is being invalidate because project cb54f85be3994a40a4b52e576b36c6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:41:43.095784 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:43.096414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-324c1c39-541d-4bc3-8229-4d631e36f6e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:43.098055 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1359/2720] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:41:42 2022] DELETE /identity/v3/projects/cb54f85be3994a40a4b52e576b36c6da => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:43.103155 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50702b47-caa6-44ff-aa95-7eb73ba30c95 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.103477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50702b47-caa6-44ff-aa95-7eb73ba30c95 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.103757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50702b47-caa6-44ff-aa95-7eb73ba30c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.122196 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-50702b47-caa6-44ff-aa95-7eb73ba30c95 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:43.123427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-50702b47-caa6-44ff-aa95-7eb73ba30c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:43.181727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1362/2721] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:43.189426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f56472-e9fc-4bd4-9c6e-c754ac5619cf None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.189720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f56472-e9fc-4bd4-9c6e-c754ac5619cf None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.190006 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f56472-e9fc-4bd4-9c6e-c754ac5619cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.208774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-e3f56472-e9fc-4bd4-9c6e-c754ac5619cf None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:43.209973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e3f56472-e9fc-4bd4-9c6e-c754ac5619cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:43.218624 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1360/2722] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:43.222584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50702b47-caa6-44ff-aa95-7eb73ba30c95 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:43.232568 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50702b47-caa6-44ff-aa95-7eb73ba30c95 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:43.237250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fe7bf-8064-46ba-ae6d-9ca349f8d96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:43.237915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fe7bf-8064-46ba-ae6d-9ca349f8d96b admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.238225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fe7bf-8064-46ba-ae6d-9ca349f8d96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.238499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fe7bf-8064-46ba-ae6d-9ca349f8d96b admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.239360 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fe7bf-8064-46ba-ae6d-9ca349f8d96b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:43.239663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fe7bf-8064-46ba-ae6d-9ca349f8d96b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:43.241849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fe7bf-8064-46ba-ae6d-9ca349f8d96b admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:43.257917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1363/2723] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:43.259049 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f56472-e9fc-4bd4-9c6e-c754ac5619cf None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:43.268444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f56472-e9fc-4bd4-9c6e-c754ac5619cf None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:43.274377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949fc09-bf12-49a2-af31-31a364a1c309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:43.275036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949fc09-bf12-49a2-af31-31a364a1c309 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.275321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949fc09-bf12-49a2-af31-31a364a1c309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.275598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949fc09-bf12-49a2-af31-31a364a1c309 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.276302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949fc09-bf12-49a2-af31-31a364a1c309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:43.278194 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949fc09-bf12-49a2-af31-31a364a1c309 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:43.308183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1361/2724] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:41:43 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:43.309380 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fe7bf-8064-46ba-ae6d-9ca349f8d96b admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:43.319589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fe7bf-8064-46ba-ae6d-9ca349f8d96b admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:43.379772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d2f44-5d74-45d1-a822-5fe47bee2870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:43.380450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d2f44-5d74-45d1-a822-5fe47bee2870 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.380736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d2f44-5d74-45d1-a822-5fe47bee2870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.381013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d2f44-5d74-45d1-a822-5fe47bee2870 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.381702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d2f44-5d74-45d1-a822-5fe47bee2870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:43.383687 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d2f44-5d74-45d1-a822-5fe47bee2870 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:43.403232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1364/2725] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:41:43 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:43.407635 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949fc09-bf12-49a2-af31-31a364a1c309 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:43.419397 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949fc09-bf12-49a2-af31-31a364a1c309 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:43.424145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4651b-0171-49a1-a95e-22df1186997a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:43.424671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4651b-0171-49a1-a95e-22df1186997a admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.424956 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4651b-0171-49a1-a95e-22df1186997a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.425241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4651b-0171-49a1-a95e-22df1186997a admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.425975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4651b-0171-49a1-a95e-22df1186997a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:43.428440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4651b-0171-49a1-a95e-22df1186997a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:43.440744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1362/2726] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:41:43.444767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d2f44-5d74-45d1-a822-5fe47bee2870 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:43.455581 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d2f44-5d74-45d1-a822-5fe47bee2870 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:43.460631 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c388f9f-c281-4887-8c3c-9074ec99d590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:43.461341 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c388f9f-c281-4887-8c3c-9074ec99d590 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.461624 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c388f9f-c281-4887-8c3c-9074ec99d590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.461939 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c388f9f-c281-4887-8c3c-9074ec99d590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.462964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c388f9f-c281-4887-8c3c-9074ec99d590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:43.464614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c388f9f-c281-4887-8c3c-9074ec99d590 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:43.475496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1365/2727] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:41:43.479609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4651b-0171-49a1-a95e-22df1186997a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:43.490648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4651b-0171-49a1-a95e-22df1186997a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:43.495771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39a1c3-849f-4050-9b7b-6c2f5457d2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:43.496665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39a1c3-849f-4050-9b7b-6c2f5457d2a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.496998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39a1c3-849f-4050-9b7b-6c2f5457d2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.497219 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39a1c3-849f-4050-9b7b-6c2f5457d2a1 admin admin] PATH_INFO: `/v3/projects/84bcc47b89c54d2f9d1fe2297c465aff/users/25d38f024efe46beaa3455abd5185ded/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.515721 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39a1c3-849f-4050-9b7b-6c2f5457d2a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84bcc47b89c54d2f9d1fe2297c465aff, user_id=25d38f024efe46beaa3455abd5185ded, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:43.518922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39a1c3-849f-4050-9b7b-6c2f5457d2a1 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:43.534803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1363/2728] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:43 2022] PUT /identity/v3/projects/84bcc47b89c54d2f9d1fe2297c465aff/users/25d38f024efe46beaa3455abd5185ded/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:43.545964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d6cda-fe67-4947-ad32-28b89ad970fd None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.546373 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d6cda-fe67-4947-ad32-28b89ad970fd None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.546741 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d6cda-fe67-4947-ad32-28b89ad970fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.575713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-b48d6cda-fe67-4947-ad32-28b89ad970fd None None] MFA Rules not processed for user `25d38f024efe46beaa3455abd5185ded`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:43.576977 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b48d6cda-fe67-4947-ad32-28b89ad970fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:43.586996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:43.656600 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1366/2729] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:43.667550 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39a1c3-849f-4050-9b7b-6c2f5457d2a1 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:43.669398 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-db39a1c3-849f-4050-9b7b-6c2f5457d2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:43.723540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:43.744493 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39a1c3-849f-4050-9b7b-6c2f5457d2a1 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:43.750199 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d913cbb-c1b4-4ad7-8e30-91bc4b484096 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:43.750875 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d913cbb-c1b4-4ad7-8e30-91bc4b484096 service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.751158 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d913cbb-c1b4-4ad7-8e30-91bc4b484096 service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.751460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d913cbb-c1b4-4ad7-8e30-91bc4b484096 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.759095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d913cbb-c1b4-4ad7-8e30-91bc4b484096 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:43.760683 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d913cbb-c1b4-4ad7-8e30-91bc4b484096 service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:43.769248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1364/2730] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:53.653739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=23 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:53.765318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=8 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:55.956865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d6cda-fe67-4947-ad32-28b89ad970fd None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:55.958314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b48d6cda-fe67-4947-ad32-28b89ad970fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:56.021915 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d6cda-fe67-4947-ad32-28b89ad970fd None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.026408 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.027271 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.027552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.027823 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] PATH_INFO: `/v3/users/01c4ed0a470f46a3b04113ee7649cb29` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.028994 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c4ed0a470f46a3b04113ee7649cb29)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.031227 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.079609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c4ed0a470f46a3b04113ee7649cb29'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.086549 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c4ed0a470f46a3b04113ee7649cb29'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.087434 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c4ed0a470f46a3b04113ee7649cb29'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.088052 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c4ed0a470f46a3b04113ee7649cb29'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.100848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c4ed0a470f46a3b04113ee7649cb29'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.113579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1367/2731] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:41:55 2022] DELETE /identity/v3/users/01c4ed0a470f46a3b04113ee7649cb29 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:56.118902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379abd55-4e84-4ac6-a3d3-897d1e59b041 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.119353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379abd55-4e84-4ac6-a3d3-897d1e59b041 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.119468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379abd55-4e84-4ac6-a3d3-897d1e59b041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.139969 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-379abd55-4e84-4ac6-a3d3-897d1e59b041 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:56.141173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-379abd55-4e84-4ac6-a3d3-897d1e59b041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:56.202652 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1365/2732] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:56.214250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:56.214250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:56.217937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379abd55-4e84-4ac6-a3d3-897d1e59b041 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:56.217937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-379abd55-4e84-4ac6-a3d3-897d1e59b041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:56.229283 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379abd55-4e84-4ac6-a3d3-897d1e59b041 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.234151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.235468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.235748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.236023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] PATH_INFO: `/v3/users/25d38f024efe46beaa3455abd5185ded` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.237247 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d38f024efe46beaa3455abd5185ded)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.239175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.274542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471a5b9-6a90-4d96-8099-366627ee0c89 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.278594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40c797-d4e6-45b0-9966-46e7c42a2142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.279245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40c797-d4e6-45b0-9966-46e7c42a2142 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.279526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40c797-d4e6-45b0-9966-46e7c42a2142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.279799 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40c797-d4e6-45b0-9966-46e7c42a2142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.289319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d38f024efe46beaa3455abd5185ded'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.289840 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40c797-d4e6-45b0-9966-46e7c42a2142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.291322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40c797-d4e6-45b0-9966-46e7c42a2142 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.301219 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d38f024efe46beaa3455abd5185ded'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.301481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1368/2733] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:56.310828 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d38f024efe46beaa3455abd5185ded'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.311459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d38f024efe46beaa3455abd5185ded'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.318768 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d38f024efe46beaa3455abd5185ded'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.324695 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1366/2734] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:41:56 2022] DELETE /identity/v3/users/25d38f024efe46beaa3455abd5185ded => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:56.329186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f41873-ec7d-4d23-9489-700311733963 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.329481 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f41873-ec7d-4d23-9489-700311733963 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.329759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f41873-ec7d-4d23-9489-700311733963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.348607 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-d8f41873-ec7d-4d23-9489-700311733963 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:56.350515 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d8f41873-ec7d-4d23-9489-700311733963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:56.369250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:56.415704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1369/2735] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:56.425506 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:56.426978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:56.487956 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c7e01-f1c5-41ec-bbae-740272cdfb54 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.491947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f75db5-b34e-438d-abbf-4e9611feb932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.492602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f75db5-b34e-438d-abbf-4e9611feb932 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.492892 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f75db5-b34e-438d-abbf-4e9611feb932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.493174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f75db5-b34e-438d-abbf-4e9611feb932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.502442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f75db5-b34e-438d-abbf-4e9611feb932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.503884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f75db5-b34e-438d-abbf-4e9611feb932 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.513810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1367/2736] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:56.630108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f41873-ec7d-4d23-9489-700311733963 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:56.631680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d8f41873-ec7d-4d23-9489-700311733963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:56.643623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f41873-ec7d-4d23-9489-700311733963 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.648992 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.649863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.650147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.650452 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] PATH_INFO: `/v3/projects/8486d3db33e445c0ba4b72ea802eb1fe` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.651681 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8486d3db33e445c0ba4b72ea802eb1fe)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.653656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.713168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8486d3db33e445c0ba4b72ea802eb1fe', 'request_id': 'req-1d68401c-786f-4025-8f21-e1a81189f988'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.723896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8486d3db33e445c0ba4b72ea802eb1fe', 'request_id': 'req-1d68401c-786f-4025-8f21-e1a81189f988'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.730588 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8486d3db33e445c0ba4b72ea802eb1fe', 'request_id': 'req-1d68401c-786f-4025-8f21-e1a81189f988'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.742450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] The token cache is being invalidate because project 8486d3db33e445c0ba4b72ea802eb1fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:41:56.742746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.743440 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.745359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1370/2737] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:41:56 2022] DELETE /identity/v3/projects/8486d3db33e445c0ba4b72ea802eb1fe => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:56.749223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f75db5-b34e-438d-abbf-4e9611feb932 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:56.759933 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f75db5-b34e-438d-abbf-4e9611feb932 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.773013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:56.821801 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.822728 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.823023 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.823305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] PATH_INFO: `/v3/users/15c3b305651d4e85a08fc242b101ad6f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.824545 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15c3b305651d4e85a08fc242b101ad6f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.826530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.877676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15c3b305651d4e85a08fc242b101ad6f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.890478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15c3b305651d4e85a08fc242b101ad6f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.899906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15c3b305651d4e85a08fc242b101ad6f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.900521 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15c3b305651d4e85a08fc242b101ad6f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.908422 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15c3b305651d4e85a08fc242b101ad6f'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:56.914780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1368/2738] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:41:56 2022] DELETE /identity/v3/users/15c3b305651d4e85a08fc242b101ad6f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:57.182920 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.185212 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:57.199814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68401c-786f-4025-8f21-e1a81189f988 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.241857 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.243287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:57.272870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.274262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.274584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.275241 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] PATH_INFO: `/v3/projects/84bcc47b89c54d2f9d1fe2297c465aff` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.276511 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84bcc47b89c54d2f9d1fe2297c465aff)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.278471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.311633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1e618-6ef7-4e14-9d0c-705063486028 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.325266 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:57.326770 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84bcc47b89c54d2f9d1fe2297c465aff', 'request_id': 'req-43bca47b-607a-48c6-abab-884580a781b0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:57.335893 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84bcc47b89c54d2f9d1fe2297c465aff', 'request_id': 'req-43bca47b-607a-48c6-abab-884580a781b0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:57.342093 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84bcc47b89c54d2f9d1fe2297c465aff', 'request_id': 'req-43bca47b-607a-48c6-abab-884580a781b0'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:57.353516 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] The token cache is being invalidate because project 84bcc47b89c54d2f9d1fe2297c465aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:41:57.353801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:57.354475 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-43bca47b-607a-48c6-abab-884580a781b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:57.354882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:57.356058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1371/2739] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:41:57 2022] DELETE /identity/v3/projects/84bcc47b89c54d2f9d1fe2297c465aff => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:57.360788 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e31c5-6dd6-4a2a-be5c-cbb67540b9cf None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.361068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e31c5-6dd6-4a2a-be5c-cbb67540b9cf None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.361339 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e31c5-6dd6-4a2a-be5c-cbb67540b9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.378438 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.378990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-002e31c5-6dd6-4a2a-be5c-cbb67540b9cf None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:57.379310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.379594 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.379871 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] PATH_INFO: `/v3/projects/71f3c77fa9b44737ae0b04f0523e384f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.380179 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-002e31c5-6dd6-4a2a-be5c-cbb67540b9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:57.381614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f3c77fa9b44737ae0b04f0523e384f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.383546 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.389323 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1372/2740] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:57.397029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc29e4e-ccb2-4b43-9945-7a901cfd16d9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.397431 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc29e4e-ccb2-4b43-9945-7a901cfd16d9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.397809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc29e4e-ccb2-4b43-9945-7a901cfd16d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.424388 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-bcc29e4e-ccb2-4b43-9945-7a901cfd16d9 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:57.425603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bcc29e4e-ccb2-4b43-9945-7a901cfd16d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:57.435657 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f3c77fa9b44737ae0b04f0523e384f', 'request_id': 'req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:57.444346 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f3c77fa9b44737ae0b04f0523e384f', 'request_id': 'req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:57.450245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f3c77fa9b44737ae0b04f0523e384f', 'request_id': 'req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:57.461728 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] The token cache is being invalidate because project 71f3c77fa9b44737ae0b04f0523e384f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:41:57.462300 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:57.462954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2bf50c48-bb41-44ea-a748-8d78b2ff6e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:57.464512 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1369/2741] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:41:57 2022] DELETE /identity/v3/projects/71f3c77fa9b44737ae0b04f0523e384f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:57.469157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259557e2-d73e-4034-becc-4f6d5ba0587e None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.469445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259557e2-d73e-4034-becc-4f6d5ba0587e None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.469723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259557e2-d73e-4034-becc-4f6d5ba0587e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.487632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-259557e2-d73e-4034-becc-4f6d5ba0587e None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:57.488897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-259557e2-d73e-4034-becc-4f6d5ba0587e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:57.489416 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1373/2742] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:41:57.494082 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc29e4e-ccb2-4b43-9945-7a901cfd16d9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.497459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1370/2743] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:41:57.506463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31ee55-ee12-4d7a-aa3b-b00930b707b9 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.506463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31ee55-ee12-4d7a-aa3b-b00930b707b9 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.506463 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31ee55-ee12-4d7a-aa3b-b00930b707b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.509168 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc29e4e-ccb2-4b43-9945-7a901cfd16d9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.518401 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aca57b-db01-4c6d-a396-33f3106f35d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.519075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aca57b-db01-4c6d-a396-33f3106f35d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.519355 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aca57b-db01-4c6d-a396-33f3106f35d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.519629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aca57b-db01-4c6d-a396-33f3106f35d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.520366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aca57b-db01-4c6d-a396-33f3106f35d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:57.520670 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aca57b-db01-4c6d-a396-33f3106f35d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.522497 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aca57b-db01-4c6d-a396-33f3106f35d4 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.525946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-4d31ee55-ee12-4d7a-aa3b-b00930b707b9 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:57.527247 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4d31ee55-ee12-4d7a-aa3b-b00930b707b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:57.538312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1374/2744] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:57.538457 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1371/2745] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:41:57.539374 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aca57b-db01-4c6d-a396-33f3106f35d4 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.540061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31ee55-ee12-4d7a-aa3b-b00930b707b9 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.549273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aca57b-db01-4c6d-a396-33f3106f35d4 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.549704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31ee55-ee12-4d7a-aa3b-b00930b707b9 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.554659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19687a57-02f2-4422-bc4d-78d7e9bbfca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.555223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b6202-0f62-4552-8e00-dad7b08eb194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.555345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19687a57-02f2-4422-bc4d-78d7e9bbfca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.555656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19687a57-02f2-4422-bc4d-78d7e9bbfca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.555940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19687a57-02f2-4422-bc4d-78d7e9bbfca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.556061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b6202-0f62-4552-8e00-dad7b08eb194 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.556292 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b6202-0f62-4552-8e00-dad7b08eb194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.556576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b6202-0f62-4552-8e00-dad7b08eb194 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.556703 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19687a57-02f2-4422-bc4d-78d7e9bbfca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.557338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b6202-0f62-4552-8e00-dad7b08eb194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:57.557649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b6202-0f62-4552-8e00-dad7b08eb194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.558661 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19687a57-02f2-4422-bc4d-78d7e9bbfca9 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.559272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b6202-0f62-4552-8e00-dad7b08eb194 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.571334 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1372/2746] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:57.575707 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b6202-0f62-4552-8e00-dad7b08eb194 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.585041 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1375/2747] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:57.585936 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b6202-0f62-4552-8e00-dad7b08eb194 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.589225 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19687a57-02f2-4422-bc4d-78d7e9bbfca9 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.643713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cd97e-c59e-48dd-8077-980849b28526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.644370 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cd97e-c59e-48dd-8077-980849b28526 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.644648 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cd97e-c59e-48dd-8077-980849b28526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.644920 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cd97e-c59e-48dd-8077-980849b28526 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.645617 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cd97e-c59e-48dd-8077-980849b28526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.647644 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cd97e-c59e-48dd-8077-980849b28526 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.654221 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19687a57-02f2-4422-bc4d-78d7e9bbfca9 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.659257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1f553-67ae-4eb4-ba55-e33acc4280fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.659932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1f553-67ae-4eb4-ba55-e33acc4280fa admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.660222 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1f553-67ae-4eb4-ba55-e33acc4280fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.660505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1f553-67ae-4eb4-ba55-e33acc4280fa admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.661200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1f553-67ae-4eb4-ba55-e33acc4280fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.663901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1f553-67ae-4eb4-ba55-e33acc4280fa admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.679466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1373/2748] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:57.683440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cd97e-c59e-48dd-8077-980849b28526 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.687482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1376/2749] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/users => generated 281 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:57.691372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1f553-67ae-4eb4-ba55-e33acc4280fa admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.700968 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1f553-67ae-4eb4-ba55-e33acc4280fa admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.718615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:57.721042 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:57.752639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cd97e-c59e-48dd-8077-980849b28526 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.757248 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9eaac-1e8a-4b49-8fef-f3e2128b8e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.757943 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9eaac-1e8a-4b49-8fef-f3e2128b8e74 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.758227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9eaac-1e8a-4b49-8fef-f3e2128b8e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.758527 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9eaac-1e8a-4b49-8fef-f3e2128b8e74 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.758680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001561d0-bdbc-4919-8547-d341bd545747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.759213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af9eaac-1e8a-4b49-8fef-f3e2128b8e74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.759342 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001561d0-bdbc-4919-8547-d341bd545747 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.759626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001561d0-bdbc-4919-8547-d341bd545747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.759899 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001561d0-bdbc-4919-8547-d341bd545747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.760613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001561d0-bdbc-4919-8547-d341bd545747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.761129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af9eaac-1e8a-4b49-8fef-f3e2128b8e74 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.762197 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001561d0-bdbc-4919-8547-d341bd545747 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.773075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1377/2750] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:41:57.777567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001561d0-bdbc-4919-8547-d341bd545747 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.784414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1374/2751] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:57.785373 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9eaac-1e8a-4b49-8fef-f3e2128b8e74 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.788706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001561d0-bdbc-4919-8547-d341bd545747 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.793616 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557890e3-da4d-48cb-b9ad-d0c998883f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.794276 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557890e3-da4d-48cb-b9ad-d0c998883f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.794595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557890e3-da4d-48cb-b9ad-d0c998883f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.794881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557890e3-da4d-48cb-b9ad-d0c998883f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.795205 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9eaac-1e8a-4b49-8fef-f3e2128b8e74 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.795656 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557890e3-da4d-48cb-b9ad-d0c998883f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.797257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557890e3-da4d-48cb-b9ad-d0c998883f26 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.799778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882d153-ceb8-45b3-9e34-b8c5ab976ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.800519 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882d153-ceb8-45b3-9e34-b8c5ab976ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.800882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882d153-ceb8-45b3-9e34-b8c5ab976ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.801241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882d153-ceb8-45b3-9e34-b8c5ab976ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.802048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882d153-ceb8-45b3-9e34-b8c5ab976ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.803680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882d153-ceb8-45b3-9e34-b8c5ab976ec6 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.811367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1378/2752] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:41:57.812493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557890e3-da4d-48cb-b9ad-d0c998883f26 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.816522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1375/2753] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:41:57.817443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882d153-ceb8-45b3-9e34-b8c5ab976ec6 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.822122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557890e3-da4d-48cb-b9ad-d0c998883f26 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.826707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd66c1f-1302-4611-9b9b-acdde09c91a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.826841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882d153-ceb8-45b3-9e34-b8c5ab976ec6 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.827619 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd66c1f-1302-4611-9b9b-acdde09c91a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.827924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd66c1f-1302-4611-9b9b-acdde09c91a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.828187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd66c1f-1302-4611-9b9b-acdde09c91a2 admin admin] PATH_INFO: `/v3/projects/186e14ebdccc4994948912d13efd0fdf/users/dd75794af1c9463a872e222d7e7fabaa/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.831319 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a9a7d-4ec1-41c9-bce7-b73cf0aad494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.831960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a9a7d-4ec1-41c9-bce7-b73cf0aad494 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.832242 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a9a7d-4ec1-41c9-bce7-b73cf0aad494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.832514 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a9a7d-4ec1-41c9-bce7-b73cf0aad494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.833232 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a9a7d-4ec1-41c9-bce7-b73cf0aad494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.835394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a9a7d-4ec1-41c9-bce7-b73cf0aad494 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.844034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd66c1f-1302-4611-9b9b-acdde09c91a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=186e14ebdccc4994948912d13efd0fdf, user_id=dd75794af1c9463a872e222d7e7fabaa, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.846759 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd66c1f-1302-4611-9b9b-acdde09c91a2 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.851804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1376/2754] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:41:57.856111 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a9a7d-4ec1-41c9-bce7-b73cf0aad494 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.861262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1379/2755] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:57 2022] PUT /identity/v3/projects/186e14ebdccc4994948912d13efd0fdf/users/dd75794af1c9463a872e222d7e7fabaa/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:57.865814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a9a7d-4ec1-41c9-bce7-b73cf0aad494 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.871300 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2677148-b3c1-4c4b-9925-3944cfac35f8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.871594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2677148-b3c1-4c4b-9925-3944cfac35f8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.871880 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2677148-b3c1-4c4b-9925-3944cfac35f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.899715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-e2677148-b3c1-4c4b-9925-3944cfac35f8 None None] MFA Rules not processed for user `dd75794af1c9463a872e222d7e7fabaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:57.900935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e2677148-b3c1-4c4b-9925-3944cfac35f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:57.922276 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd60b8-bcdb-4217-86de-2240bfd02df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.923190 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd60b8-bcdb-4217-86de-2240bfd02df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.923474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd60b8-bcdb-4217-86de-2240bfd02df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.923747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd60b8-bcdb-4217-86de-2240bfd02df0 admin admin] PATH_INFO: `/v3/projects/df1977060ca348ca8ce5d46aec8d1696/users/857bba849829435e9b37cead7c6cf20e/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.937583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd60b8-bcdb-4217-86de-2240bfd02df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df1977060ca348ca8ce5d46aec8d1696, user_id=857bba849829435e9b37cead7c6cf20e, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.940328 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd60b8-bcdb-4217-86de-2240bfd02df0 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.954552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1377/2756] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:57 2022] PUT /identity/v3/projects/df1977060ca348ca8ce5d46aec8d1696/users/857bba849829435e9b37cead7c6cf20e/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:57.966989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1380/2757] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:41:57.971771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd60b8-bcdb-4217-86de-2240bfd02df0 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.014609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f840bc6-11f2-4a63-9ada-22bfd28366e5 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.014816 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f840bc6-11f2-4a63-9ada-22bfd28366e5 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.015312 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f840bc6-11f2-4a63-9ada-22bfd28366e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.035517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd60b8-bcdb-4217-86de-2240bfd02df0 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.040671 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daf2cf-e767-4690-a6f2-80881030d80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.041320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daf2cf-e767-4690-a6f2-80881030d80b admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.041608 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daf2cf-e767-4690-a6f2-80881030d80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.041907 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daf2cf-e767-4690-a6f2-80881030d80b admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.042588 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-9f840bc6-11f2-4a63-9ada-22bfd28366e5 None None] MFA Rules not processed for user `857bba849829435e9b37cead7c6cf20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:58.042681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4daf2cf-e767-4690-a6f2-80881030d80b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.043772 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-9f840bc6-11f2-4a63-9ada-22bfd28366e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:58.044503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4daf2cf-e767-4690-a6f2-80881030d80b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.072170 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1378/2758] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:58.073498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daf2cf-e767-4690-a6f2-80881030d80b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.081904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:58.110396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1381/2759] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:41:58.114368 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f840bc6-11f2-4a63-9ada-22bfd28366e5 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.121126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:58.125470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f840bc6-11f2-4a63-9ada-22bfd28366e5 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.139443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daf2cf-e767-4690-a6f2-80881030d80b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.144106 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da39cb0-8277-4480-b9b3-b82a5f3f6b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.144776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da39cb0-8277-4480-b9b3-b82a5f3f6b35 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.145064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da39cb0-8277-4480-b9b3-b82a5f3f6b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.145337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da39cb0-8277-4480-b9b3-b82a5f3f6b35 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.146555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da39cb0-8277-4480-b9b3-b82a5f3f6b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.148531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da39cb0-8277-4480-b9b3-b82a5f3f6b35 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.171774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1379/2760] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:41:58 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:58.173274 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da39cb0-8277-4480-b9b3-b82a5f3f6b35 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.182778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da39cb0-8277-4480-b9b3-b82a5f3f6b35 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.183460 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c65fe-c4fb-4a49-a786-9bacf1da4257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.184118 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c65fe-c4fb-4a49-a786-9bacf1da4257 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.184397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c65fe-c4fb-4a49-a786-9bacf1da4257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.184665 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c65fe-c4fb-4a49-a786-9bacf1da4257 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.185345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c65fe-c4fb-4a49-a786-9bacf1da4257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.187293 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c65fe-c4fb-4a49-a786-9bacf1da4257 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.187447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed094a-fe22-42e2-8e46-64da1ee1b7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.188213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed094a-fe22-42e2-8e46-64da1ee1b7ed admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.188508 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed094a-fe22-42e2-8e46-64da1ee1b7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.188793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed094a-fe22-42e2-8e46-64da1ee1b7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.189520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed094a-fe22-42e2-8e46-64da1ee1b7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.191179 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed094a-fe22-42e2-8e46-64da1ee1b7ed admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.202270 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1380/2761] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:41:58.206906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed094a-fe22-42e2-8e46-64da1ee1b7ed admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.218781 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1382/2762] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:41:58 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:58.218932 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed094a-fe22-42e2-8e46-64da1ee1b7ed admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.220063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c65fe-c4fb-4a49-a786-9bacf1da4257 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.274548 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09db0c-2237-4479-9b1f-208b570c1039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.275378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09db0c-2237-4479-9b1f-208b570c1039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.275666 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09db0c-2237-4479-9b1f-208b570c1039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.275949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09db0c-2237-4479-9b1f-208b570c1039 admin admin] PATH_INFO: `/v3/projects/1132f80e7ff542ddb38ae514ee259b9e/users/0550a615ec20446995b536776478b4d9/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.281892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c65fe-c4fb-4a49-a786-9bacf1da4257 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.286796 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b1067-a882-4d29-a7c7-5bca21a80d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.287457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1067-a882-4d29-a7c7-5bca21a80d7e admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.287738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1067-a882-4d29-a7c7-5bca21a80d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.288019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1067-a882-4d29-a7c7-5bca21a80d7e admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.288708 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b1067-a882-4d29-a7c7-5bca21a80d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.290793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b1067-a882-4d29-a7c7-5bca21a80d7e admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.290909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09db0c-2237-4479-9b1f-208b570c1039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1132f80e7ff542ddb38ae514ee259b9e, user_id=0550a615ec20446995b536776478b4d9, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.293753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09db0c-2237-4479-9b1f-208b570c1039 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.307502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1381/2763] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:58 2022] PUT /identity/v3/projects/1132f80e7ff542ddb38ae514ee259b9e/users/0550a615ec20446995b536776478b4d9/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:58.311134 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1383/2764] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:41:58 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:58.311455 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09db0c-2237-4479-9b1f-208b570c1039 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.314785 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b1067-a882-4d29-a7c7-5bca21a80d7e admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.324841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b1067-a882-4d29-a7c7-5bca21a80d7e admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.372788 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09db0c-2237-4479-9b1f-208b570c1039 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.377783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ace54-0035-44d5-8d4a-a4458369edee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.378488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ace54-0035-44d5-8d4a-a4458369edee admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.378781 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ace54-0035-44d5-8d4a-a4458369edee admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.379064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ace54-0035-44d5-8d4a-a4458369edee admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.379791 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ace54-0035-44d5-8d4a-a4458369edee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.380237 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185640f-c0eb-4026-933f-8067dd7148dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.381423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185640f-c0eb-4026-933f-8067dd7148dd admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.381713 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185640f-c0eb-4026-933f-8067dd7148dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.381926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ace54-0035-44d5-8d4a-a4458369edee admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.382024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185640f-c0eb-4026-933f-8067dd7148dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.382729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185640f-c0eb-4026-933f-8067dd7148dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.384985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185640f-c0eb-4026-933f-8067dd7148dd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.391939 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1382/2765] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:41:58.396008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ace54-0035-44d5-8d4a-a4458369edee admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.398904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1384/2766] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:41:58.399800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185640f-c0eb-4026-933f-8067dd7148dd admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.403178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:58.407230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ace54-0035-44d5-8d4a-a4458369edee admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.409793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185640f-c0eb-4026-933f-8067dd7148dd admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.413174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b285d39-6568-4e8a-b5bf-5be13ca03c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.414089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b285d39-6568-4e8a-b5bf-5be13ca03c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.414471 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b285d39-6568-4e8a-b5bf-5be13ca03c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.414750 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b285d39-6568-4e8a-b5bf-5be13ca03c14 admin admin] PATH_INFO: `/v3/projects/1132f80e7ff542ddb38ae514ee259b9e/users/0550a615ec20446995b536776478b4d9/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.415019 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0e71b-a14c-4941-ac47-58c87bdb1b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.415903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0e71b-a14c-4941-ac47-58c87bdb1b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.416190 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0e71b-a14c-4941-ac47-58c87bdb1b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.416470 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0e71b-a14c-4941-ac47-58c87bdb1b09 admin admin] PATH_INFO: `/v3/projects/3639a50db21d4bad9dbe104bba0d6701/users/d7330e4553e04f06830aafda78efd53d/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.416763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b285d39-6568-4e8a-b5bf-5be13ca03c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1132f80e7ff542ddb38ae514ee259b9e, user_id=0550a615ec20446995b536776478b4d9, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.419466 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b285d39-6568-4e8a-b5bf-5be13ca03c14 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.429104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:58.429957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1383/2767] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:58 2022] PUT /identity/v3/projects/1132f80e7ff542ddb38ae514ee259b9e/users/0550a615ec20446995b536776478b4d9/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:58.437257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0e71b-a14c-4941-ac47-58c87bdb1b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3639a50db21d4bad9dbe104bba0d6701, user_id=d7330e4553e04f06830aafda78efd53d, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.440334 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0e71b-a14c-4941-ac47-58c87bdb1b09 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.441257 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc15d15-9762-4a0a-bc3f-041434da6b8d None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.441552 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc15d15-9762-4a0a-bc3f-041434da6b8d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.441834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc15d15-9762-4a0a-bc3f-041434da6b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.455215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1385/2768] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:58 2022] PUT /identity/v3/projects/3639a50db21d4bad9dbe104bba0d6701/users/d7330e4553e04f06830aafda78efd53d/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:58.459682 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0e71b-a14c-4941-ac47-58c87bdb1b09 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.473340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-bbc15d15-9762-4a0a-bc3f-041434da6b8d None None] MFA Rules not processed for user `0550a615ec20446995b536776478b4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:58.474556 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-bbc15d15-9762-4a0a-bc3f-041434da6b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:58.525847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0e71b-a14c-4941-ac47-58c87bdb1b09 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.531720 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918477ea-2a17-49ce-b223-e5daaf1b8801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.532462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918477ea-2a17-49ce-b223-e5daaf1b8801 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.533013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918477ea-2a17-49ce-b223-e5daaf1b8801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.533329 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918477ea-2a17-49ce-b223-e5daaf1b8801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.534187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918477ea-2a17-49ce-b223-e5daaf1b8801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.535824 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918477ea-2a17-49ce-b223-e5daaf1b8801 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.542553 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1384/2769] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:41:58.546842 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1386/2770] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:41:58.550126 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc15d15-9762-4a0a-bc3f-041434da6b8d None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.554868 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918477ea-2a17-49ce-b223-e5daaf1b8801 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.556518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-918477ea-2a17-49ce-b223-e5daaf1b8801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:58.561009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc15d15-9762-4a0a-bc3f-041434da6b8d None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.565609 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33622da9-eab3-4439-930d-45c33ca75e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.566516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33622da9-eab3-4439-930d-45c33ca75e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.566796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33622da9-eab3-4439-930d-45c33ca75e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.567071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33622da9-eab3-4439-930d-45c33ca75e8b admin admin] PATH_INFO: `/v3/projects/3639a50db21d4bad9dbe104bba0d6701/users/d7330e4553e04f06830aafda78efd53d/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.569558 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33622da9-eab3-4439-930d-45c33ca75e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3639a50db21d4bad9dbe104bba0d6701, user_id=d7330e4553e04f06830aafda78efd53d, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.572200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33622da9-eab3-4439-930d-45c33ca75e8b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.582445 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1385/2771] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:41:58 2022] PUT /identity/v3/projects/3639a50db21d4bad9dbe104bba0d6701/users/d7330e4553e04f06830aafda78efd53d/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:58.593486 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a5aba-9615-4452-a09a-9ff4b572abda None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.593771 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a5aba-9615-4452-a09a-9ff4b572abda None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.594044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a5aba-9615-4452-a09a-9ff4b572abda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.616669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918477ea-2a17-49ce-b223-e5daaf1b8801 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.620537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-573a5aba-9615-4452-a09a-9ff4b572abda None None] MFA Rules not processed for user `d7330e4553e04f06830aafda78efd53d`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:58.621728 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-573a5aba-9615-4452-a09a-9ff4b572abda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:58.665659 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fe4a9-b8d3-4e11-974e-b92aadda6154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.666039 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe4a9-b8d3-4e11-974e-b92aadda6154 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.666347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe4a9-b8d3-4e11-974e-b92aadda6154 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.666622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe4a9-b8d3-4e11-974e-b92aadda6154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.689134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1386/2772] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:41:58.700620 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a5aba-9615-4452-a09a-9ff4b572abda None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.701986 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-573a5aba-9615-4452-a09a-9ff4b572abda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:58.713976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a5aba-9615-4452-a09a-9ff4b572abda None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.717632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29e3f9-24e5-4dfe-9176-49fc6fa55ea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.718296 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29e3f9-24e5-4dfe-9176-49fc6fa55ea7 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.718597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29e3f9-24e5-4dfe-9176-49fc6fa55ea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.718873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29e3f9-24e5-4dfe-9176-49fc6fa55ea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.724972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:58.727962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29e3f9-24e5-4dfe-9176-49fc6fa55ea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.729425 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29e3f9-24e5-4dfe-9176-49fc6fa55ea7 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.729771 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fe4a9-b8d3-4e11-974e-b92aadda6154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.731455 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fe4a9-b8d3-4e11-974e-b92aadda6154 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.738910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1387/2773] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:41:58.742214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1387/2774] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:41:58.770139 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29e3f9-24e5-4dfe-9176-49fc6fa55ea7 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.819372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fe4a9-b8d3-4e11-974e-b92aadda6154 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.822667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:58.828238 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fe4a9-b8d3-4e11-974e-b92aadda6154 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.832060 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84047b90-15b0-4953-b4cf-f70b3717998a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.832753 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84047b90-15b0-4953-b4cf-f70b3717998a service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.832987 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84047b90-15b0-4953-b4cf-f70b3717998a service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.833264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84047b90-15b0-4953-b4cf-f70b3717998a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.847436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29e3f9-24e5-4dfe-9176-49fc6fa55ea7 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.852312 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.853326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.853612 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.853888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] PATH_INFO: `/v3/users/857bba849829435e9b37cead7c6cf20e` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.855120 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] RBAC: Authorizing `identity:delete_user(user_id=857bba849829435e9b37cead7c6cf20e)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.857245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.894579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84047b90-15b0-4953-b4cf-f70b3717998a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.896086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84047b90-15b0-4953-b4cf-f70b3717998a service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.903719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '857bba849829435e9b37cead7c6cf20e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:58.905029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1388/2775] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:41:58.917141 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '857bba849829435e9b37cead7c6cf20e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:58.924831 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '857bba849829435e9b37cead7c6cf20e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:58.925488 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '857bba849829435e9b37cead7c6cf20e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:58.933849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '857bba849829435e9b37cead7c6cf20e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:58.940284 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1388/2776] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:41:58 2022] DELETE /identity/v3/users/857bba849829435e9b37cead7c6cf20e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:58.944990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80a4b-d376-4456-a4a7-6059e0764d4b None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.945428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80a4b-d376-4456-a4a7-6059e0764d4b None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.945716 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80a4b-d376-4456-a4a7-6059e0764d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.965031 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-bbe80a4b-d376-4456-a4a7-6059e0764d4b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:58.966370 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bbe80a4b-d376-4456-a4a7-6059e0764d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:59.033870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1389/2777] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:41:59.045975 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:59.047424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:59.106855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33117dbe-1f03-4a3e-abb9-53bc5594067d admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:59.110468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c0d35-fd32-43a8-a37f-321fd4570598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:59.111118 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c0d35-fd32-43a8-a37f-321fd4570598 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:59.111405 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c0d35-fd32-43a8-a37f-321fd4570598 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:59.111677 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c0d35-fd32-43a8-a37f-321fd4570598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:59.121209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c0d35-fd32-43a8-a37f-321fd4570598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:59.122684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c0d35-fd32-43a8-a37f-321fd4570598 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:59.131783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1389/2778] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:41:59.387072 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe80a4b-d376-4456-a4a7-6059e0764d4b None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:59.389085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bbe80a4b-d376-4456-a4a7-6059e0764d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:59.401838 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe80a4b-d376-4456-a4a7-6059e0764d4b None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:59.406763 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:59.407753 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:59.408036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:59.408307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] PATH_INFO: `/v3/projects/df1977060ca348ca8ce5d46aec8d1696` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:59.409514 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df1977060ca348ca8ce5d46aec8d1696)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:59.411586 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:59.456595 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df1977060ca348ca8ce5d46aec8d1696', 'request_id': 'req-479359fa-425a-4b43-9c95-6888d82f0e58'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:59.464364 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df1977060ca348ca8ce5d46aec8d1696', 'request_id': 'req-479359fa-425a-4b43-9c95-6888d82f0e58'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:59.469755 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df1977060ca348ca8ce5d46aec8d1696', 'request_id': 'req-479359fa-425a-4b43-9c95-6888d82f0e58'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:59.481009 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] The token cache is being invalidate because project df1977060ca348ca8ce5d46aec8d1696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:41:59.481290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:59.481932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:59.483441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1390/2779] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:41:59 2022] DELETE /identity/v3/projects/df1977060ca348ca8ce5d46aec8d1696 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:59.487412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c0d35-fd32-43a8-a37f-321fd4570598 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:59.496891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c0d35-fd32-43a8-a37f-321fd4570598 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:59.546004 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:59.552575 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8151f5-0dd7-4e40-8ba8-bc10300f3cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:59.553675 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8151f5-0dd7-4e40-8ba8-bc10300f3cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:59.554015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8151f5-0dd7-4e40-8ba8-bc10300f3cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:59.554502 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8151f5-0dd7-4e40-8ba8-bc10300f3cda admin admin] PATH_INFO: `/v3/users/d7330e4553e04f06830aafda78efd53d` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:59.556093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8151f5-0dd7-4e40-8ba8-bc10300f3cda admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7330e4553e04f06830aafda78efd53d)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:59.558406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8151f5-0dd7-4e40-8ba8-bc10300f3cda admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:59.607168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0c8151f5-0dd7-4e40-8ba8-bc10300f3cda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7330e4553e04f06830aafda78efd53d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:59.619354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0c8151f5-0dd7-4e40-8ba8-bc10300f3cda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7330e4553e04f06830aafda78efd53d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:59.626629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0c8151f5-0dd7-4e40-8ba8-bc10300f3cda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7330e4553e04f06830aafda78efd53d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:59.627289 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0c8151f5-0dd7-4e40-8ba8-bc10300f3cda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7330e4553e04f06830aafda78efd53d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:59.634150 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-0c8151f5-0dd7-4e40-8ba8-bc10300f3cda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7330e4553e04f06830aafda78efd53d'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:41:59.640036 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1390/2780] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:41:59 2022] DELETE /identity/v3/users/d7330e4553e04f06830aafda78efd53d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:59.910121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:59.911597 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:59.924697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479359fa-425a-4b43-9c95-6888d82f0e58 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:59.937861 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:59.984952 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:59.985810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:59.986089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:59.986393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] PATH_INFO: `/v3/projects/3639a50db21d4bad9dbe104bba0d6701` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:59.987560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3639a50db21d4bad9dbe104bba0d6701)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:59.989478 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.034533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3639a50db21d4bad9dbe104bba0d6701', 'request_id': 'req-b61091f6-1f7a-4365-943c-62c2a6730b02'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:00.041429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3639a50db21d4bad9dbe104bba0d6701', 'request_id': 'req-b61091f6-1f7a-4365-943c-62c2a6730b02'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:00.047234 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3639a50db21d4bad9dbe104bba0d6701', 'request_id': 'req-b61091f6-1f7a-4365-943c-62c2a6730b02'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:00.062256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] The token cache is being invalidate because project 3639a50db21d4bad9dbe104bba0d6701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:00.062649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:00.063371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b61091f6-1f7a-4365-943c-62c2a6730b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:00.065303 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1391/2781] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:41:59 2022] DELETE /identity/v3/projects/3639a50db21d4bad9dbe104bba0d6701 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:00.070459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a61f3-d297-4af9-b593-5df6e6f50987 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.070751 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a61f3-d297-4af9-b593-5df6e6f50987 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.071031 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a61f3-d297-4af9-b593-5df6e6f50987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.089632 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-fc4a61f3-d297-4af9-b593-5df6e6f50987 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:00.090870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-fc4a61f3-d297-4af9-b593-5df6e6f50987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:00.153093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1391/2782] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:00.160964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481aae9-3ad0-47f9-b8ae-ebcf782644f9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.161259 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481aae9-3ad0-47f9-b8ae-ebcf782644f9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.161559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481aae9-3ad0-47f9-b8ae-ebcf782644f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.182801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7481aae9-3ad0-47f9-b8ae-ebcf782644f9 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:00.182801 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7481aae9-3ad0-47f9-b8ae-ebcf782644f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:00.190708 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1392/2783] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:00.195444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a61f3-d297-4af9-b593-5df6e6f50987 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.205480 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a61f3-d297-4af9-b593-5df6e6f50987 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.210736 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976ec97-623b-41f3-aba8-d5f83c4ba8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.211401 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976ec97-623b-41f3-aba8-d5f83c4ba8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.211779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976ec97-623b-41f3-aba8-d5f83c4ba8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.212325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976ec97-623b-41f3-aba8-d5f83c4ba8c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.213100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976ec97-623b-41f3-aba8-d5f83c4ba8c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:00.213408 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976ec97-623b-41f3-aba8-d5f83c4ba8c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.215048 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976ec97-623b-41f3-aba8-d5f83c4ba8c8 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.228110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1392/2784] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:00.232467 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481aae9-3ad0-47f9-b8ae-ebcf782644f9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.242784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481aae9-3ad0-47f9-b8ae-ebcf782644f9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.247680 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4131a1-2e6e-402c-84a6-1bec678385ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.248444 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4131a1-2e6e-402c-84a6-1bec678385ff admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.248757 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4131a1-2e6e-402c-84a6-1bec678385ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.249016 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4131a1-2e6e-402c-84a6-1bec678385ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.249765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4131a1-2e6e-402c-84a6-1bec678385ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.251695 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4131a1-2e6e-402c-84a6-1bec678385ff admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.281128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1393/2785] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:42:00 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:00.282465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976ec97-623b-41f3-aba8-d5f83c4ba8c8 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.292359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976ec97-623b-41f3-aba8-d5f83c4ba8c8 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.352564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc428cb-54af-422a-b5a1-1f7e2addb772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.353251 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc428cb-54af-422a-b5a1-1f7e2addb772 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.353537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc428cb-54af-422a-b5a1-1f7e2addb772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.353816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc428cb-54af-422a-b5a1-1f7e2addb772 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.354559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc428cb-54af-422a-b5a1-1f7e2addb772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.356474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc428cb-54af-422a-b5a1-1f7e2addb772 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.366564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:00.380267 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1393/2786] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:42:00 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:42:00.384800 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4131a1-2e6e-402c-84a6-1bec678385ff admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.396946 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4131a1-2e6e-402c-84a6-1bec678385ff admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.402095 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243fa8b-94b9-402e-95cf-e06f47ebff99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.402839 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243fa8b-94b9-402e-95cf-e06f47ebff99 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.403125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243fa8b-94b9-402e-95cf-e06f47ebff99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.403425 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243fa8b-94b9-402e-95cf-e06f47ebff99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.404141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243fa8b-94b9-402e-95cf-e06f47ebff99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.405867 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243fa8b-94b9-402e-95cf-e06f47ebff99 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.417071 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1394/2787] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:00.421227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc428cb-54af-422a-b5a1-1f7e2addb772 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.433740 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc428cb-54af-422a-b5a1-1f7e2addb772 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.437541 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3b8f1-7989-4cb7-94a0-82dd0e65e768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.438216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3b8f1-7989-4cb7-94a0-82dd0e65e768 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.438543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3b8f1-7989-4cb7-94a0-82dd0e65e768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.438852 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3b8f1-7989-4cb7-94a0-82dd0e65e768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.439583 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3b8f1-7989-4cb7-94a0-82dd0e65e768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.441223 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3b8f1-7989-4cb7-94a0-82dd0e65e768 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.451890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1394/2788] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:00.456384 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243fa8b-94b9-402e-95cf-e06f47ebff99 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.467114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243fa8b-94b9-402e-95cf-e06f47ebff99 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.471707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba781a8a-9c89-4bdf-a8f1-2adb6281f3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.472587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba781a8a-9c89-4bdf-a8f1-2adb6281f3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.472870 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba781a8a-9c89-4bdf-a8f1-2adb6281f3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.473150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba781a8a-9c89-4bdf-a8f1-2adb6281f3ea admin admin] PATH_INFO: `/v3/projects/75294eafddc547caaaac2be9fa215bf2/users/39d9abbdef6c42e3ab5adfbaf470fee4/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.492698 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba781a8a-9c89-4bdf-a8f1-2adb6281f3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=75294eafddc547caaaac2be9fa215bf2, user_id=39d9abbdef6c42e3ab5adfbaf470fee4, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.495406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba781a8a-9c89-4bdf-a8f1-2adb6281f3ea admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.509567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1395/2789] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:00 2022] PUT /identity/v3/projects/75294eafddc547caaaac2be9fa215bf2/users/39d9abbdef6c42e3ab5adfbaf470fee4/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:00.521176 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cacb21-6fef-4deb-8324-885d3f6f987b None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.521681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cacb21-6fef-4deb-8324-885d3f6f987b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.521766 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cacb21-6fef-4deb-8324-885d3f6f987b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.548972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a5cacb21-6fef-4deb-8324-885d3f6f987b None None] MFA Rules not processed for user `39d9abbdef6c42e3ab5adfbaf470fee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:00.550189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a5cacb21-6fef-4deb-8324-885d3f6f987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:00.616966 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1395/2790] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:00.621417 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba781a8a-9c89-4bdf-a8f1-2adb6281f3ea admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.664872 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:00.683601 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba781a8a-9c89-4bdf-a8f1-2adb6281f3ea admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.688234 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7957988-f54f-4852-941d-4e773050620d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.688890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7957988-f54f-4852-941d-4e773050620d admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.689181 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7957988-f54f-4852-941d-4e773050620d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.689458 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7957988-f54f-4852-941d-4e773050620d admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.690156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7957988-f54f-4852-941d-4e773050620d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.692103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7957988-f54f-4852-941d-4e773050620d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.717426 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1396/2791] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:42:00 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:42:00.721629 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cacb21-6fef-4deb-8324-885d3f6f987b None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.732972 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cacb21-6fef-4deb-8324-885d3f6f987b None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.789011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512caea-feeb-4855-92a0-0facf6f31a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.789687 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512caea-feeb-4855-92a0-0facf6f31a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.789992 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512caea-feeb-4855-92a0-0facf6f31a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.790258 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512caea-feeb-4855-92a0-0facf6f31a29 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.790985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512caea-feeb-4855-92a0-0facf6f31a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.792905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512caea-feeb-4855-92a0-0facf6f31a29 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.812071 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1396/2792] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:42:00 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:00.816045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7957988-f54f-4852-941d-4e773050620d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.826567 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7957988-f54f-4852-941d-4e773050620d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.831171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad43cdf-2842-4c46-9698-2cc69f5da093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.831825 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad43cdf-2842-4c46-9698-2cc69f5da093 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.832111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad43cdf-2842-4c46-9698-2cc69f5da093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.832390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad43cdf-2842-4c46-9698-2cc69f5da093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.833108 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad43cdf-2842-4c46-9698-2cc69f5da093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.835263 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad43cdf-2842-4c46-9698-2cc69f5da093 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.845736 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1397/2793] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:00.850085 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512caea-feeb-4855-92a0-0facf6f31a29 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.860063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512caea-feeb-4855-92a0-0facf6f31a29 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.864689 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76fa10-7148-45ff-96ac-4824e72e7701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.865841 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76fa10-7148-45ff-96ac-4824e72e7701 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.866128 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76fa10-7148-45ff-96ac-4824e72e7701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.866443 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76fa10-7148-45ff-96ac-4824e72e7701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.867173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76fa10-7148-45ff-96ac-4824e72e7701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.868764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76fa10-7148-45ff-96ac-4824e72e7701 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.878562 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1397/2794] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:00.883038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad43cdf-2842-4c46-9698-2cc69f5da093 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.892802 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad43cdf-2842-4c46-9698-2cc69f5da093 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.899144 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e1005-634d-4a77-bbfe-41efbe05f21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.900024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e1005-634d-4a77-bbfe-41efbe05f21c admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.900306 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e1005-634d-4a77-bbfe-41efbe05f21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.900579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e1005-634d-4a77-bbfe-41efbe05f21c admin admin] PATH_INFO: `/v3/projects/68415b9d92fb44759ef4dc7584eb70d5/users/7ad5a8ace9f9498db894a06ec4638506/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.920477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e1005-634d-4a77-bbfe-41efbe05f21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=68415b9d92fb44759ef4dc7584eb70d5, user_id=7ad5a8ace9f9498db894a06ec4638506, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.923150 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e1005-634d-4a77-bbfe-41efbe05f21c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.936178 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1398/2795] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:00 2022] PUT /identity/v3/projects/68415b9d92fb44759ef4dc7584eb70d5/users/7ad5a8ace9f9498db894a06ec4638506/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:00.947062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227ef6f-5a79-4414-8b35-503268a5ddd1 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.947459 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227ef6f-5a79-4414-8b35-503268a5ddd1 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.947805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227ef6f-5a79-4414-8b35-503268a5ddd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.974452 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-7227ef6f-5a79-4414-8b35-503268a5ddd1 None None] MFA Rules not processed for user `7ad5a8ace9f9498db894a06ec4638506`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:00.975663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7227ef6f-5a79-4414-8b35-503268a5ddd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:01.015101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:01.039678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1398/2796] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:01.044220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e1005-634d-4a77-bbfe-41efbe05f21c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.106676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e1005-634d-4a77-bbfe-41efbe05f21c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.111261 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc9228-c459-4d4d-a704-ae8be6042755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.111907 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc9228-c459-4d4d-a704-ae8be6042755 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.112191 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc9228-c459-4d4d-a704-ae8be6042755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.112462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc9228-c459-4d4d-a704-ae8be6042755 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.113170 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc9228-c459-4d4d-a704-ae8be6042755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.115664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc9228-c459-4d4d-a704-ae8be6042755 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.140195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1399/2797] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:42:01 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:01.144293 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227ef6f-5a79-4414-8b35-503268a5ddd1 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.154947 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227ef6f-5a79-4414-8b35-503268a5ddd1 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.209062 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a263a-7c04-4eae-b943-da820cd11f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.209667 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a263a-7c04-4eae-b943-da820cd11f3a admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.209957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a263a-7c04-4eae-b943-da820cd11f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.210236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a263a-7c04-4eae-b943-da820cd11f3a admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.210973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a263a-7c04-4eae-b943-da820cd11f3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.213151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a263a-7c04-4eae-b943-da820cd11f3a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.232730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1399/2798] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:42:01 2022] POST /identity/v3/users => generated 299 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:42:01.236646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc9228-c459-4d4d-a704-ae8be6042755 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.247584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc9228-c459-4d4d-a704-ae8be6042755 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.252620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03d5a6-e3ba-49f6-9316-8e61636d6b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.253322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03d5a6-e3ba-49f6-9316-8e61636d6b89 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.253613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03d5a6-e3ba-49f6-9316-8e61636d6b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.253910 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03d5a6-e3ba-49f6-9316-8e61636d6b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.254664 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03d5a6-e3ba-49f6-9316-8e61636d6b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.256308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03d5a6-e3ba-49f6-9316-8e61636d6b89 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.267820 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1400/2799] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:01.272343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a263a-7c04-4eae-b943-da820cd11f3a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.282308 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a263a-7c04-4eae-b943-da820cd11f3a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.287383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76ae5e-01f5-435b-a9d5-497e2e65fbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.288272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76ae5e-01f5-435b-a9d5-497e2e65fbfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.288543 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76ae5e-01f5-435b-a9d5-497e2e65fbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.288819 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76ae5e-01f5-435b-a9d5-497e2e65fbfa admin admin] PATH_INFO: `/v3/projects/3492baf8273c4076b4489b62084781f9/users/eaecddecaeda4c65ae3a1cbaa2f7a889/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.303793 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76ae5e-01f5-435b-a9d5-497e2e65fbfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3492baf8273c4076b4489b62084781f9, user_id=eaecddecaeda4c65ae3a1cbaa2f7a889, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.306477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76ae5e-01f5-435b-a9d5-497e2e65fbfa admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.319262 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1400/2800] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:01 2022] PUT /identity/v3/projects/3492baf8273c4076b4489b62084781f9/users/eaecddecaeda4c65ae3a1cbaa2f7a889/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:01.323082 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03d5a6-e3ba-49f6-9316-8e61636d6b89 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.332922 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03d5a6-e3ba-49f6-9316-8e61636d6b89 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.389848 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d5b65-580b-4226-9efd-25e51893b529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.390554 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d5b65-580b-4226-9efd-25e51893b529 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.390840 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d5b65-580b-4226-9efd-25e51893b529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.391115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d5b65-580b-4226-9efd-25e51893b529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.391836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d5b65-580b-4226-9efd-25e51893b529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.393441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d5b65-580b-4226-9efd-25e51893b529 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.403931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1401/2801] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:01.408474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76ae5e-01f5-435b-a9d5-497e2e65fbfa admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.419089 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76ae5e-01f5-435b-a9d5-497e2e65fbfa admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.423764 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a382485-418a-433d-9616-7ef72f2dc5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.424646 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a382485-418a-433d-9616-7ef72f2dc5be admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.424937 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a382485-418a-433d-9616-7ef72f2dc5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.425216 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a382485-418a-433d-9616-7ef72f2dc5be admin admin] PATH_INFO: `/v3/projects/3492baf8273c4076b4489b62084781f9/users/eaecddecaeda4c65ae3a1cbaa2f7a889/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.427329 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a382485-418a-433d-9616-7ef72f2dc5be admin admin] RBAC: Authorizing `identity:create_grant(project_id=3492baf8273c4076b4489b62084781f9, user_id=eaecddecaeda4c65ae3a1cbaa2f7a889, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.429973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a382485-418a-433d-9616-7ef72f2dc5be admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.440193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1401/2802] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:01 2022] PUT /identity/v3/projects/3492baf8273c4076b4489b62084781f9/users/eaecddecaeda4c65ae3a1cbaa2f7a889/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:01.451730 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42327e9-e0c5-491d-ad42-7e621b4ab767 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.452024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42327e9-e0c5-491d-ad42-7e621b4ab767 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.452331 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42327e9-e0c5-491d-ad42-7e621b4ab767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.458198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:01.480357 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-a42327e9-e0c5-491d-ad42-7e621b4ab767 None None] MFA Rules not processed for user `eaecddecaeda4c65ae3a1cbaa2f7a889`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:01.481491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a42327e9-e0c5-491d-ad42-7e621b4ab767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:01.552024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1402/2803] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:01.563254 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a382485-418a-433d-9616-7ef72f2dc5be admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.565164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1a382485-418a-433d-9616-7ef72f2dc5be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:01.631738 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a382485-418a-433d-9616-7ef72f2dc5be admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.637286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e072264-1a30-42d1-9aa6-04b58525f403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.637286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e072264-1a30-42d1-9aa6-04b58525f403 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.637286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e072264-1a30-42d1-9aa6-04b58525f403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.637286 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e072264-1a30-42d1-9aa6-04b58525f403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.646446 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e072264-1a30-42d1-9aa6-04b58525f403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.647948 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e072264-1a30-42d1-9aa6-04b58525f403 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.654378 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42327e9-e0c5-491d-ad42-7e621b4ab767 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.655971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a42327e9-e0c5-491d-ad42-7e621b4ab767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:01.658461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1402/2804] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:01.722702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42327e9-e0c5-491d-ad42-7e621b4ab767 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.727793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.730877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.731122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.731411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] PATH_INFO: `/v3/users/dd75794af1c9463a872e222d7e7fabaa` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.732695 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd75794af1c9463a872e222d7e7fabaa)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.735023 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.783979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd75794af1c9463a872e222d7e7fabaa'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:01.792363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd75794af1c9463a872e222d7e7fabaa'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:01.793045 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd75794af1c9463a872e222d7e7fabaa'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:01.793684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd75794af1c9463a872e222d7e7fabaa'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:01.806955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd75794af1c9463a872e222d7e7fabaa'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:01.819280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1403/2805] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:01 2022] DELETE /identity/v3/users/dd75794af1c9463a872e222d7e7fabaa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:01.823623 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee79fe5-7fb4-4c05-9ba7-20710a4a6fda None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.823914 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee79fe5-7fb4-4c05-9ba7-20710a4a6fda None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.824200 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee79fe5-7fb4-4c05-9ba7-20710a4a6fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.844123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-3ee79fe5-7fb4-4c05-9ba7-20710a4a6fda None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:01.845309 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3ee79fe5-7fb4-4c05-9ba7-20710a4a6fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:01.878101 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:01.909802 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1403/2806] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:01.920400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.921832 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:01.931629 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:01.987882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0d2b7-820c-4ce0-829b-c4d970a50996 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.991543 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d856-eaa8-4098-bf4c-b8f90ae4c1e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.992193 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d856-eaa8-4098-bf4c-b8f90ae4c1e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.992474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d856-eaa8-4098-bf4c-b8f90ae4c1e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.992746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d856-eaa8-4098-bf4c-b8f90ae4c1e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.002574 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89d856-eaa8-4098-bf4c-b8f90ae4c1e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:02.004056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89d856-eaa8-4098-bf4c-b8f90ae4c1e9 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:02.014133 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1404/2807] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:42:02.170723 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee79fe5-7fb4-4c05-9ba7-20710a4a6fda None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:02.172113 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3ee79fe5-7fb4-4c05-9ba7-20710a4a6fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:02.230691 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee79fe5-7fb4-4c05-9ba7-20710a4a6fda None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:02.234157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caf229-5d14-4a7d-9d43-f7e5727667fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:02.234954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caf229-5d14-4a7d-9d43-f7e5727667fa service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.235103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caf229-5d14-4a7d-9d43-f7e5727667fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.235377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caf229-5d14-4a7d-9d43-f7e5727667fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.237162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b3caf229-5d14-4a7d-9d43-f7e5727667fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:02.289095 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d856-eaa8-4098-bf4c-b8f90ae4c1e9 service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:02.290702 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5d89d856-eaa8-4098-bf4c-b8f90ae4c1e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:02.299921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3caf229-5d14-4a7d-9d43-f7e5727667fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:02.301376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3caf229-5d14-4a7d-9d43-f7e5727667fa service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:02.309424 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d856-eaa8-4098-bf4c-b8f90ae4c1e9 service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:02.314090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:02.314983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.315307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.315536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] PATH_INFO: `/v3/projects/186e14ebdccc4994948912d13efd0fdf` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.316673 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1404/2808] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:02.316849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=186e14ebdccc4994948912d13efd0fdf)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:02.319403 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:02.368768 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '186e14ebdccc4994948912d13efd0fdf', 'request_id': 'req-81163979-f763-49c8-b0a0-8311b9d50865'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.376006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '186e14ebdccc4994948912d13efd0fdf', 'request_id': 'req-81163979-f763-49c8-b0a0-8311b9d50865'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.387004 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '186e14ebdccc4994948912d13efd0fdf', 'request_id': 'req-81163979-f763-49c8-b0a0-8311b9d50865'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.398427 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] The token cache is being invalidate because project 186e14ebdccc4994948912d13efd0fdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:02.398721 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.399376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.401036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1405/2809] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:02 2022] DELETE /identity/v3/projects/186e14ebdccc4994948912d13efd0fdf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:42:02.405043 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caf229-5d14-4a7d-9d43-f7e5727667fa service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:02.416184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caf229-5d14-4a7d-9d43-f7e5727667fa service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:02.477726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b3a86-499c-4fd2-8c2d-b09c9ef547d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:02.478713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b3a86-499c-4fd2-8c2d-b09c9ef547d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.479001 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b3a86-499c-4fd2-8c2d-b09c9ef547d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.479282 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b3a86-499c-4fd2-8c2d-b09c9ef547d5 admin admin] PATH_INFO: `/v3/users/0550a615ec20446995b536776478b4d9` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.480487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b3a86-499c-4fd2-8c2d-b09c9ef547d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0550a615ec20446995b536776478b4d9)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:02.482474 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b3a86-499c-4fd2-8c2d-b09c9ef547d5 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:02.518674 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:02.533525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cf1b3a86-499c-4fd2-8c2d-b09c9ef547d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0550a615ec20446995b536776478b4d9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.546918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cf1b3a86-499c-4fd2-8c2d-b09c9ef547d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0550a615ec20446995b536776478b4d9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.555839 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cf1b3a86-499c-4fd2-8c2d-b09c9ef547d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0550a615ec20446995b536776478b4d9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.556589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cf1b3a86-499c-4fd2-8c2d-b09c9ef547d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0550a615ec20446995b536776478b4d9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.565461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-cf1b3a86-499c-4fd2-8c2d-b09c9ef547d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0550a615ec20446995b536776478b4d9'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.572433 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1405/2810] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:02 2022] DELETE /identity/v3/users/0550a615ec20446995b536776478b4d9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:02.851129 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:02.853135 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:02.867033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81163979-f763-49c8-b0a0-8311b9d50865 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:02.923369 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:02.924230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.924516 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.924790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] PATH_INFO: `/v3/projects/1132f80e7ff542ddb38ae514ee259b9e` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.925975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1132f80e7ff542ddb38ae514ee259b9e)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:02.933112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:02.969850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:02.982180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1132f80e7ff542ddb38ae514ee259b9e', 'request_id': 'req-05cc6946-6717-4734-8366-ae67208abaaa'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.991018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1132f80e7ff542ddb38ae514ee259b9e', 'request_id': 'req-05cc6946-6717-4734-8366-ae67208abaaa'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:02.996535 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1132f80e7ff542ddb38ae514ee259b9e', 'request_id': 'req-05cc6946-6717-4734-8366-ae67208abaaa'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:03.007283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] The token cache is being invalidate because project 1132f80e7ff542ddb38ae514ee259b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:03.007572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:03.008257 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-05cc6946-6717-4734-8366-ae67208abaaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:03.009805 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1406/2811] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:02 2022] DELETE /identity/v3/projects/1132f80e7ff542ddb38ae514ee259b9e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:03.015720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa953118-2bc9-41ce-98bd-54c7711d4ce2 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.016018 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa953118-2bc9-41ce-98bd-54c7711d4ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.016299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa953118-2bc9-41ce-98bd-54c7711d4ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.033894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-aa953118-2bc9-41ce-98bd-54c7711d4ce2 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:03.035067 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-aa953118-2bc9-41ce-98bd-54c7711d4ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:03.095637 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1406/2812] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:03.103983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3083e-59d1-45e4-85c9-4ef11579c6fc None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.104302 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3083e-59d1-45e4-85c9-4ef11579c6fc None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.104580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3083e-59d1-45e4-85c9-4ef11579c6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.122451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-14a3083e-59d1-45e4-85c9-4ef11579c6fc None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:03.123636 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-14a3083e-59d1-45e4-85c9-4ef11579c6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:03.131392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1407/2813] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:42:03.135387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa953118-2bc9-41ce-98bd-54c7711d4ce2 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.144891 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa953118-2bc9-41ce-98bd-54c7711d4ce2 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.150140 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c4dca-2db7-4989-84c7-59ed44442840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.151451 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c4dca-2db7-4989-84c7-59ed44442840 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.151741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c4dca-2db7-4989-84c7-59ed44442840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.152019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c4dca-2db7-4989-84c7-59ed44442840 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.152748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c4dca-2db7-4989-84c7-59ed44442840 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:03.153059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c4dca-2db7-4989-84c7-59ed44442840 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.154690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c4dca-2db7-4989-84c7-59ed44442840 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.166589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1407/2814] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:03.170712 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3083e-59d1-45e4-85c9-4ef11579c6fc None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.179920 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3083e-59d1-45e4-85c9-4ef11579c6fc None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.184590 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603cac8-9c9e-4d94-a8ed-2f8c16ffc644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.185247 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603cac8-9c9e-4d94-a8ed-2f8c16ffc644 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.185532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603cac8-9c9e-4d94-a8ed-2f8c16ffc644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.185812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603cac8-9c9e-4d94-a8ed-2f8c16ffc644 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.186532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603cac8-9c9e-4d94-a8ed-2f8c16ffc644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.188450 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603cac8-9c9e-4d94-a8ed-2f8c16ffc644 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.215701 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1408/2815] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:42:03 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:03.219957 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c4dca-2db7-4989-84c7-59ed44442840 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.230708 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c4dca-2db7-4989-84c7-59ed44442840 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.288779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0646ff0-105c-42f8-ab2e-29d4e8e8a9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.289347 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0646ff0-105c-42f8-ab2e-29d4e8e8a9cc admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.289631 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0646ff0-105c-42f8-ab2e-29d4e8e8a9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.289905 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0646ff0-105c-42f8-ab2e-29d4e8e8a9cc admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.290627 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0646ff0-105c-42f8-ab2e-29d4e8e8a9cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.292532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0646ff0-105c-42f8-ab2e-29d4e8e8a9cc admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.311446 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1408/2816] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:42:03 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:42:03.315532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603cac8-9c9e-4d94-a8ed-2f8c16ffc644 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.325677 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603cac8-9c9e-4d94-a8ed-2f8c16ffc644 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.330247 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0276a-fb6b-48a1-921c-0c00897f32b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.330927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0276a-fb6b-48a1-921c-0c00897f32b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.331213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0276a-fb6b-48a1-921c-0c00897f32b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.331493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0276a-fb6b-48a1-921c-0c00897f32b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.332220 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0276a-fb6b-48a1-921c-0c00897f32b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.334376 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0276a-fb6b-48a1-921c-0c00897f32b4 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.344711 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1409/2817] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:03.348275 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0646ff0-105c-42f8-ab2e-29d4e8e8a9cc admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.359019 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0646ff0-105c-42f8-ab2e-29d4e8e8a9cc admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.363693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e3b9c-85f4-49bf-bf04-6f6bea00dab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.364352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3b9c-85f4-49bf-bf04-6f6bea00dab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.364647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3b9c-85f4-49bf-bf04-6f6bea00dab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.364922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3b9c-85f4-49bf-bf04-6f6bea00dab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.365634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e3b9c-85f4-49bf-bf04-6f6bea00dab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.367245 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e3b9c-85f4-49bf-bf04-6f6bea00dab0 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.376749 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1409/2818] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:03.380620 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0276a-fb6b-48a1-921c-0c00897f32b4 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.390509 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0276a-fb6b-48a1-921c-0c00897f32b4 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.395152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31265c-e63f-477b-8e88-6ed00ed3d262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.396536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31265c-e63f-477b-8e88-6ed00ed3d262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.396817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31265c-e63f-477b-8e88-6ed00ed3d262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.397089 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31265c-e63f-477b-8e88-6ed00ed3d262 admin admin] PATH_INFO: `/v3/projects/71042f5eb4a44b05b16660f67f0d5c62/users/4c2e29b1d180458885824334ebf28d1e/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.411477 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31265c-e63f-477b-8e88-6ed00ed3d262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71042f5eb4a44b05b16660f67f0d5c62, user_id=4c2e29b1d180458885824334ebf28d1e, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.414795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31265c-e63f-477b-8e88-6ed00ed3d262 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.427433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1410/2819] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:03 2022] PUT /identity/v3/projects/71042f5eb4a44b05b16660f67f0d5c62/users/4c2e29b1d180458885824334ebf28d1e/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:42:03.483598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c4265-5b7e-4289-941d-c762c8e48dce None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.483908 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c4265-5b7e-4289-941d-c762c8e48dce None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.484148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c4265-5b7e-4289-941d-c762c8e48dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.510436 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-691c4265-5b7e-4289-941d-c762c8e48dce None None] MFA Rules not processed for user `4c2e29b1d180458885824334ebf28d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:03.511619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-691c4265-5b7e-4289-941d-c762c8e48dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:03.515505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:03.577791 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1410/2820] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:03.582328 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31265c-e63f-477b-8e88-6ed00ed3d262 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.650184 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31265c-e63f-477b-8e88-6ed00ed3d262 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.655034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc698bf-a351-49d0-825c-6f237f5b3b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.655817 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc698bf-a351-49d0-825c-6f237f5b3b96 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.656109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc698bf-a351-49d0-825c-6f237f5b3b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.656386 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc698bf-a351-49d0-825c-6f237f5b3b96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.657078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc698bf-a351-49d0-825c-6f237f5b3b96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.659074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc698bf-a351-49d0-825c-6f237f5b3b96 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.687902 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1411/2821] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:42:03 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:03.689236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c4265-5b7e-4289-941d-c762c8e48dce None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.700399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c4265-5b7e-4289-941d-c762c8e48dce None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.757720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563df1b6-6625-4a56-a536-777ec0578db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.758525 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563df1b6-6625-4a56-a536-777ec0578db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.758824 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563df1b6-6625-4a56-a536-777ec0578db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.759124 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563df1b6-6625-4a56-a536-777ec0578db3 admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.759899 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563df1b6-6625-4a56-a536-777ec0578db3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.761847 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563df1b6-6625-4a56-a536-777ec0578db3 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.783697 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1411/2822] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:42:03 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:03.789324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc698bf-a351-49d0-825c-6f237f5b3b96 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.800572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc698bf-a351-49d0-825c-6f237f5b3b96 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.805986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd043c4f-d455-4cb8-aef5-7dc36eadc42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.806726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd043c4f-d455-4cb8-aef5-7dc36eadc42c admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.807009 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd043c4f-d455-4cb8-aef5-7dc36eadc42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.807288 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd043c4f-d455-4cb8-aef5-7dc36eadc42c admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.808106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd043c4f-d455-4cb8-aef5-7dc36eadc42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.809617 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd043c4f-d455-4cb8-aef5-7dc36eadc42c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.824502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1412/2823] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:03.828835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563df1b6-6625-4a56-a536-777ec0578db3 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.843688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563df1b6-6625-4a56-a536-777ec0578db3 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.848414 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4b9c4-d635-41b6-93d0-9943d05e7bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.849326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4b9c4-d635-41b6-93d0-9943d05e7bde admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.849618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4b9c4-d635-41b6-93d0-9943d05e7bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.849902 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4b9c4-d635-41b6-93d0-9943d05e7bde admin admin] PATH_INFO: `/v3/projects/89d576f45ca648e48a5d85dc4df48a17/users/023ac7d003684130a39bcc097388cde8/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.865295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4b9c4-d635-41b6-93d0-9943d05e7bde admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d576f45ca648e48a5d85dc4df48a17, user_id=023ac7d003684130a39bcc097388cde8, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.867999 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4b9c4-d635-41b6-93d0-9943d05e7bde admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.884125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1412/2824] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:03 2022] PUT /identity/v3/projects/89d576f45ca648e48a5d85dc4df48a17/users/023ac7d003684130a39bcc097388cde8/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:03.888206 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd043c4f-d455-4cb8-aef5-7dc36eadc42c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.902602 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd043c4f-d455-4cb8-aef5-7dc36eadc42c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.929981 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:03.959028 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1ab80-10b2-43b9-a0b2-69e05e0c548e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.959706 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1ab80-10b2-43b9-a0b2-69e05e0c548e admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.960026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1ab80-10b2-43b9-a0b2-69e05e0c548e admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.960337 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1ab80-10b2-43b9-a0b2-69e05e0c548e admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.961063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1ab80-10b2-43b9-a0b2-69e05e0c548e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.962989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1ab80-10b2-43b9-a0b2-69e05e0c548e admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.973014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1413/2825] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:03.977647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4b9c4-d635-41b6-93d0-9943d05e7bde admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.988896 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4b9c4-d635-41b6-93d0-9943d05e7bde admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.994317 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6d6fa-8000-4f68-a055-cbd77591e61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.995238 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6d6fa-8000-4f68-a055-cbd77591e61e admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.996260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6d6fa-8000-4f68-a055-cbd77591e61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.996561 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6d6fa-8000-4f68-a055-cbd77591e61e admin admin] PATH_INFO: `/v3/projects/89d576f45ca648e48a5d85dc4df48a17/users/023ac7d003684130a39bcc097388cde8/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.999532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6d6fa-8000-4f68-a055-cbd77591e61e admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d576f45ca648e48a5d85dc4df48a17, user_id=023ac7d003684130a39bcc097388cde8, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:04.002785 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6d6fa-8000-4f68-a055-cbd77591e61e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:04.014533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1413/2826] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:03 2022] PUT /identity/v3/projects/89d576f45ca648e48a5d85dc4df48a17/users/023ac7d003684130a39bcc097388cde8/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:04.024435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48885b-d298-4fdc-9c55-25ac40894b37 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:04.024731 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48885b-d298-4fdc-9c55-25ac40894b37 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:04.025013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48885b-d298-4fdc-9c55-25ac40894b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:04.050493 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-ec48885b-d298-4fdc-9c55-25ac40894b37 None None] MFA Rules not processed for user `023ac7d003684130a39bcc097388cde8`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:04.051714 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-ec48885b-d298-4fdc-9c55-25ac40894b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:04.118372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1414/2827] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:04.128842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6d6fa-8000-4f68-a055-cbd77591e61e admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:04.131140 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-46f6d6fa-8000-4f68-a055-cbd77591e61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:04.189230 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6d6fa-8000-4f68-a055-cbd77591e61e admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:04.194307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877f394-4df0-4062-80cc-f9f142825d8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:04.195000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877f394-4df0-4062-80cc-f9f142825d8a service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:04.195256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877f394-4df0-4062-80cc-f9f142825d8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:04.195530 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877f394-4df0-4062-80cc-f9f142825d8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:04.204864 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877f394-4df0-4062-80cc-f9f142825d8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:04.206279 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877f394-4df0-4062-80cc-f9f142825d8a service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:04.211195 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:04.216614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1414/2828] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:04.566055 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48885b-d298-4fdc-9c55-25ac40894b37 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:04.626245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48885b-d298-4fdc-9c55-25ac40894b37 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:04.630729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:04.631588 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:04.631896 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:04.632171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] PATH_INFO: `/v3/users/4c2e29b1d180458885824334ebf28d1e` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:04.633367 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2e29b1d180458885824334ebf28d1e)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:04.635315 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:04.681333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2e29b1d180458885824334ebf28d1e'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:04.688491 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2e29b1d180458885824334ebf28d1e'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:04.689228 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2e29b1d180458885824334ebf28d1e'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:04.689833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2e29b1d180458885824334ebf28d1e'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:04.691739 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:04.703704 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2e29b1d180458885824334ebf28d1e'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:04.716633 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1415/2829] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:04 2022] DELETE /identity/v3/users/4c2e29b1d180458885824334ebf28d1e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:04.721083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34412a6-3e1d-421f-8932-c37d6f8c7d3b None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:04.721526 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34412a6-3e1d-421f-8932-c37d6f8c7d3b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:04.721645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34412a6-3e1d-421f-8932-c37d6f8c7d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:04.739377 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-c34412a6-3e1d-421f-8932-c37d6f8c7d3b None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:04.740545 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c34412a6-3e1d-421f-8932-c37d6f8c7d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:04.797945 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1415/2830] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:04.807326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:04.808813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:04.868662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43368d-f271-47d6-aef9-7021a827108b admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:04.872589 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f67d12-ec91-4d92-baef-a04918bc33ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:04.873247 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f67d12-ec91-4d92-baef-a04918bc33ae service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:04.873540 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f67d12-ec91-4d92-baef-a04918bc33ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:04.873874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f67d12-ec91-4d92-baef-a04918bc33ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:04.883471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f67d12-ec91-4d92-baef-a04918bc33ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:04.884859 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f67d12-ec91-4d92-baef-a04918bc33ae service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:04.894634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1416/2831] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:05.177973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34412a6-3e1d-421f-8932-c37d6f8c7d3b None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:05.179454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-c34412a6-3e1d-421f-8932-c37d6f8c7d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:05.191188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34412a6-3e1d-421f-8932-c37d6f8c7d3b None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:05.196374 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:05.197239 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:05.197630 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:05.197951 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] PATH_INFO: `/v3/projects/71042f5eb4a44b05b16660f67f0d5c62` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:05.199566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71042f5eb4a44b05b16660f67f0d5c62)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:05.201504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:05.250147 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71042f5eb4a44b05b16660f67f0d5c62', 'request_id': 'req-b279e105-8ee2-4f85-bc11-655e8e3e6a99'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.257888 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71042f5eb4a44b05b16660f67f0d5c62', 'request_id': 'req-b279e105-8ee2-4f85-bc11-655e8e3e6a99'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.263344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71042f5eb4a44b05b16660f67f0d5c62', 'request_id': 'req-b279e105-8ee2-4f85-bc11-655e8e3e6a99'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.274574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] The token cache is being invalidate because project 71042f5eb4a44b05b16660f67f0d5c62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:05.274859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.275477 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.276996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1416/2832] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:05 2022] DELETE /identity/v3/projects/71042f5eb4a44b05b16660f67f0d5c62 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:05.280986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f67d12-ec91-4d92-baef-a04918bc33ae service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:05.290505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f67d12-ec91-4d92-baef-a04918bc33ae service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:05.344841 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3689cd-f117-47b1-a815-5b0890da2739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:05.345690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3689cd-f117-47b1-a815-5b0890da2739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:05.345971 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3689cd-f117-47b1-a815-5b0890da2739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:05.346241 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3689cd-f117-47b1-a815-5b0890da2739 admin admin] PATH_INFO: `/v3/users/023ac7d003684130a39bcc097388cde8` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:05.347441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3689cd-f117-47b1-a815-5b0890da2739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=023ac7d003684130a39bcc097388cde8)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:05.349354 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3689cd-f117-47b1-a815-5b0890da2739 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:05.394689 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1d3689cd-f117-47b1-a815-5b0890da2739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '023ac7d003684130a39bcc097388cde8'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.401831 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1d3689cd-f117-47b1-a815-5b0890da2739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '023ac7d003684130a39bcc097388cde8'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.402507 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1d3689cd-f117-47b1-a815-5b0890da2739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '023ac7d003684130a39bcc097388cde8'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.403175 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1d3689cd-f117-47b1-a815-5b0890da2739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '023ac7d003684130a39bcc097388cde8'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.415140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-1d3689cd-f117-47b1-a815-5b0890da2739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '023ac7d003684130a39bcc097388cde8'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.420200 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:05.430669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1417/2833] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:05 2022] DELETE /identity/v3/users/023ac7d003684130a39bcc097388cde8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:05.706117 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:05.707567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:05.721705 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e105-8ee2-4f85-bc11-655e8e3e6a99 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:05.781312 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:05.782177 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:05.782498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:05.782779 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] PATH_INFO: `/v3/projects/89d576f45ca648e48a5d85dc4df48a17` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:05.783960 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89d576f45ca648e48a5d85dc4df48a17)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:05.785859 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:05.817173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:05.834653 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89d576f45ca648e48a5d85dc4df48a17', 'request_id': 'req-7db2a4aa-9320-455a-9bd6-8278c627d731'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.841867 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89d576f45ca648e48a5d85dc4df48a17', 'request_id': 'req-7db2a4aa-9320-455a-9bd6-8278c627d731'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.847424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89d576f45ca648e48a5d85dc4df48a17', 'request_id': 'req-7db2a4aa-9320-455a-9bd6-8278c627d731'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.858984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] The token cache is being invalidate because project 89d576f45ca648e48a5d85dc4df48a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:05.859260 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.859890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-7db2a4aa-9320-455a-9bd6-8278c627d731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:05.861444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1417/2834] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:05 2022] DELETE /identity/v3/projects/89d576f45ca648e48a5d85dc4df48a17 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:05.866998 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987daef-26fc-4849-a270-939537b0cf30 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:05.867290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987daef-26fc-4849-a270-939537b0cf30 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:05.867587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987daef-26fc-4849-a270-939537b0cf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:05.885240 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8987daef-26fc-4849-a270-939537b0cf30 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:05.886462 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8987daef-26fc-4849-a270-939537b0cf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:05.942548 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1418/2835] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:05.950323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160636fc-8f3a-496d-8bb0-5b3223f5bf89 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:05.950566 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160636fc-8f3a-496d-8bb0-5b3223f5bf89 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:05.950846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160636fc-8f3a-496d-8bb0-5b3223f5bf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:05.968944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-160636fc-8f3a-496d-8bb0-5b3223f5bf89 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:05.970129 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-160636fc-8f3a-496d-8bb0-5b3223f5bf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:05.978444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1418/2836] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:05.982390 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987daef-26fc-4849-a270-939537b0cf30 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:05.992317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987daef-26fc-4849-a270-939537b0cf30 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:05.996812 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e26aba-0d56-4d74-bb8d-c2bfd553242c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:05.997453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e26aba-0d56-4d74-bb8d-c2bfd553242c admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:05.997733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e26aba-0d56-4d74-bb8d-c2bfd553242c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:05.998006 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e26aba-0d56-4d74-bb8d-c2bfd553242c admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:05.998767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e26aba-0d56-4d74-bb8d-c2bfd553242c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:05.999074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e26aba-0d56-4d74-bb8d-c2bfd553242c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.000675 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e26aba-0d56-4d74-bb8d-c2bfd553242c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.011785 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1419/2837] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:06.015803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160636fc-8f3a-496d-8bb0-5b3223f5bf89 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.025222 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160636fc-8f3a-496d-8bb0-5b3223f5bf89 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.029756 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed7a1b-7f79-4344-87a3-2aaa672eacc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.030435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed7a1b-7f79-4344-87a3-2aaa672eacc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.030717 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed7a1b-7f79-4344-87a3-2aaa672eacc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.030988 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed7a1b-7f79-4344-87a3-2aaa672eacc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.031665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed7a1b-7f79-4344-87a3-2aaa672eacc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.033571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed7a1b-7f79-4344-87a3-2aaa672eacc7 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.059020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1419/2838] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:42:06 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:06.063022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e26aba-0d56-4d74-bb8d-c2bfd553242c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.072888 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e26aba-0d56-4d74-bb8d-c2bfd553242c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.128635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a2bd5-fa1e-4377-93b6-9096fc1b06dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.129295 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a2bd5-fa1e-4377-93b6-9096fc1b06dc admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.129579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a2bd5-fa1e-4377-93b6-9096fc1b06dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.129860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a2bd5-fa1e-4377-93b6-9096fc1b06dc admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.130588 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a2bd5-fa1e-4377-93b6-9096fc1b06dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.133278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a2bd5-fa1e-4377-93b6-9096fc1b06dc admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.153695 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1420/2839] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:42:06 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:42:06.157856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed7a1b-7f79-4344-87a3-2aaa672eacc7 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.168926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed7a1b-7f79-4344-87a3-2aaa672eacc7 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.173605 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192d94c-b9f2-42f9-b514-ab891804db6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.174262 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192d94c-b9f2-42f9-b514-ab891804db6e admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.174576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192d94c-b9f2-42f9-b514-ab891804db6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.174850 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192d94c-b9f2-42f9-b514-ab891804db6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.175897 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192d94c-b9f2-42f9-b514-ab891804db6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.177533 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192d94c-b9f2-42f9-b514-ab891804db6e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.191210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1420/2840] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:06.192394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a2bd5-fa1e-4377-93b6-9096fc1b06dc admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.202522 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a2bd5-fa1e-4377-93b6-9096fc1b06dc admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.207095 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccb433-2c3e-49d2-a253-a17dae36d0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.207753 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccb433-2c3e-49d2-a253-a17dae36d0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.208037 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccb433-2c3e-49d2-a253-a17dae36d0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.208840 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccb433-2c3e-49d2-a253-a17dae36d0d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.209564 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccb433-2c3e-49d2-a253-a17dae36d0d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.211186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccb433-2c3e-49d2-a253-a17dae36d0d8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.222669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1421/2841] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:06.231963 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192d94c-b9f2-42f9-b514-ab891804db6e admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.237298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192d94c-b9f2-42f9-b514-ab891804db6e admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.242044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8012c-2997-478c-84ac-e6bae7bcb215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.242954 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8012c-2997-478c-84ac-e6bae7bcb215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.243235 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8012c-2997-478c-84ac-e6bae7bcb215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.243506 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8012c-2997-478c-84ac-e6bae7bcb215 admin admin] PATH_INFO: `/v3/projects/39462a6ef7e5432597f3f83f5916e09a/users/8237fb1c731d4917b2b825fbc0aadcdf/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.259255 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8012c-2997-478c-84ac-e6bae7bcb215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39462a6ef7e5432597f3f83f5916e09a, user_id=8237fb1c731d4917b2b825fbc0aadcdf, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.261894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8012c-2997-478c-84ac-e6bae7bcb215 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.281876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1421/2842] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:06 2022] PUT /identity/v3/projects/39462a6ef7e5432597f3f83f5916e09a/users/8237fb1c731d4917b2b825fbc0aadcdf/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:06.287580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958999-fede-40d6-8a70-8d5683962cb9 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.287890 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958999-fede-40d6-8a70-8d5683962cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.288167 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958999-fede-40d6-8a70-8d5683962cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.314945 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8b958999-fede-40d6-8a70-8d5683962cb9 None None] MFA Rules not processed for user `8237fb1c731d4917b2b825fbc0aadcdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:06.316163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8b958999-fede-40d6-8a70-8d5683962cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:06.341295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8012c-2997-478c-84ac-e6bae7bcb215 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.342990 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a9d8012c-2997-478c-84ac-e6bae7bcb215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:06.367067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:06.384809 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1422/2843] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:06.389074 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b958999-fede-40d6-8a70-8d5683962cb9 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.407326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8012c-2997-478c-84ac-e6bae7bcb215 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.411034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbd24a-ff8c-4f59-be85-85b9157d281e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.412360 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbd24a-ff8c-4f59-be85-85b9157d281e service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.412643 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbd24a-ff8c-4f59-be85-85b9157d281e service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.413174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbd24a-ff8c-4f59-be85-85b9157d281e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.415340 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1fcbd24a-ff8c-4f59-be85-85b9157d281e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:06.450346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b958999-fede-40d6-8a70-8d5683962cb9 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.455278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382fcb7-f9e7-47a4-8782-8ddf21d73a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.456139 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382fcb7-f9e7-47a4-8782-8ddf21d73a6b admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.456301 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382fcb7-f9e7-47a4-8782-8ddf21d73a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.456488 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382fcb7-f9e7-47a4-8782-8ddf21d73a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.457218 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382fcb7-f9e7-47a4-8782-8ddf21d73a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.459307 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382fcb7-f9e7-47a4-8782-8ddf21d73a6b admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.478003 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbd24a-ff8c-4f59-be85-85b9157d281e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.479506 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbd24a-ff8c-4f59-be85-85b9157d281e service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.489123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1423/2844] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:42:06 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:42:06.489322 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1422/2845] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:42:06.490526 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382fcb7-f9e7-47a4-8782-8ddf21d73a6b admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.553275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382fcb7-f9e7-47a4-8782-8ddf21d73a6b admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.558482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16816a-d31b-440b-b4ce-48b734b23484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.559173 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16816a-d31b-440b-b4ce-48b734b23484 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.559482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16816a-d31b-440b-b4ce-48b734b23484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.559754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16816a-d31b-440b-b4ce-48b734b23484 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.560437 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16816a-d31b-440b-b4ce-48b734b23484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.562443 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16816a-d31b-440b-b4ce-48b734b23484 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.584983 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1424/2846] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:42:06 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:06.589838 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbd24a-ff8c-4f59-be85-85b9157d281e service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.596944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:06.602432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbd24a-ff8c-4f59-be85-85b9157d281e service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.608505 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b3519-edc1-4e64-bc60-8483ea91f688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.609406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b3519-edc1-4e64-bc60-8483ea91f688 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.610010 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b3519-edc1-4e64-bc60-8483ea91f688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.610305 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b3519-edc1-4e64-bc60-8483ea91f688 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.611057 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b3519-edc1-4e64-bc60-8483ea91f688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.612645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b3519-edc1-4e64-bc60-8483ea91f688 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.622872 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1423/2847] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:06.627171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16816a-d31b-440b-b4ce-48b734b23484 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.638991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16816a-d31b-440b-b4ce-48b734b23484 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.643609 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd35bd8-1719-4dff-ab8f-2c07e0f77679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.644484 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd35bd8-1719-4dff-ab8f-2c07e0f77679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.644765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd35bd8-1719-4dff-ab8f-2c07e0f77679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.645038 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd35bd8-1719-4dff-ab8f-2c07e0f77679 admin admin] PATH_INFO: `/v3/projects/32c53bafea0149ec9258d9d0d6e3de7f/users/6d27f7287fd44535a54f7b984ab1a7dd/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.661103 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd35bd8-1719-4dff-ab8f-2c07e0f77679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c53bafea0149ec9258d9d0d6e3de7f, user_id=6d27f7287fd44535a54f7b984ab1a7dd, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.664021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd35bd8-1719-4dff-ab8f-2c07e0f77679 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.677035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1425/2848] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:06 2022] PUT /identity/v3/projects/32c53bafea0149ec9258d9d0d6e3de7f/users/6d27f7287fd44535a54f7b984ab1a7dd/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:06.681278 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b3519-edc1-4e64-bc60-8483ea91f688 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.692489 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b3519-edc1-4e64-bc60-8483ea91f688 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.754277 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a25af2-41fd-4226-b594-0ddd365123b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.755614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25af2-41fd-4226-b594-0ddd365123b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.755909 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25af2-41fd-4226-b594-0ddd365123b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.756185 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25af2-41fd-4226-b594-0ddd365123b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.756906 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a25af2-41fd-4226-b594-0ddd365123b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.759794 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a25af2-41fd-4226-b594-0ddd365123b6 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.772132 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1424/2849] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:06.776232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd35bd8-1719-4dff-ab8f-2c07e0f77679 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.787393 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd35bd8-1719-4dff-ab8f-2c07e0f77679 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:06.792275 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd84813-dc6c-4dcc-81e5-240f73ed477e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:06.793235 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd84813-dc6c-4dcc-81e5-240f73ed477e admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.793591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd84813-dc6c-4dcc-81e5-240f73ed477e admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.793940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd84813-dc6c-4dcc-81e5-240f73ed477e admin admin] PATH_INFO: `/v3/projects/32c53bafea0149ec9258d9d0d6e3de7f/users/6d27f7287fd44535a54f7b984ab1a7dd/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.796030 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd84813-dc6c-4dcc-81e5-240f73ed477e admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c53bafea0149ec9258d9d0d6e3de7f, user_id=6d27f7287fd44535a54f7b984ab1a7dd, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:06.798846 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd84813-dc6c-4dcc-81e5-240f73ed477e admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:06.812100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1426/2850] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:06 2022] PUT /identity/v3/projects/32c53bafea0149ec9258d9d0d6e3de7f/users/6d27f7287fd44535a54f7b984ab1a7dd/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:06.823344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b683a85-2af1-41da-8a9a-8db1e9a0da14 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.823633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b683a85-2af1-41da-8a9a-8db1e9a0da14 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.824327 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b683a85-2af1-41da-8a9a-8db1e9a0da14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.855131 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-9b683a85-2af1-41da-8a9a-8db1e9a0da14 None None] MFA Rules not processed for user `6d27f7287fd44535a54f7b984ab1a7dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:06.856402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9b683a85-2af1-41da-8a9a-8db1e9a0da14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:06.927547 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1425/2851] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:06.939975 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd84813-dc6c-4dcc-81e5-240f73ed477e admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:06.972563 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:06.999577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd84813-dc6c-4dcc-81e5-240f73ed477e admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:07.003506 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda57d5-fea2-41b7-a59b-0b4b993264da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:07.004180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda57d5-fea2-41b7-a59b-0b4b993264da service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.004463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda57d5-fea2-41b7-a59b-0b4b993264da service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.004737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda57d5-fea2-41b7-a59b-0b4b993264da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.061305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda57d5-fea2-41b7-a59b-0b4b993264da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:07.062858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda57d5-fea2-41b7-a59b-0b4b993264da service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:07.072399 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1427/2852] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:07.718108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b683a85-2af1-41da-8a9a-8db1e9a0da14 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:07.719518 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9b683a85-2af1-41da-8a9a-8db1e9a0da14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:07.782694 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b683a85-2af1-41da-8a9a-8db1e9a0da14 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:07.787986 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:07.788865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.789151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.789428 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] PATH_INFO: `/v3/users/39d9abbdef6c42e3ab5adfbaf470fee4` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.790658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d9abbdef6c42e3ab5adfbaf470fee4)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:07.792638 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:07.808044 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:07.837658 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d9abbdef6c42e3ab5adfbaf470fee4'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:07.849679 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d9abbdef6c42e3ab5adfbaf470fee4'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:07.856702 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d9abbdef6c42e3ab5adfbaf470fee4'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:07.857332 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d9abbdef6c42e3ab5adfbaf470fee4'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:07.864337 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d9abbdef6c42e3ab5adfbaf470fee4'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:07.870090 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1426/2853] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:07 2022] DELETE /identity/v3/users/39d9abbdef6c42e3ab5adfbaf470fee4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:07.874708 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88ac05-6199-4aa7-8650-5ca88793a660 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.875029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88ac05-6199-4aa7-8650-5ca88793a660 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.875338 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88ac05-6199-4aa7-8650-5ca88793a660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.891904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-bc88ac05-6199-4aa7-8650-5ca88793a660 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:07.893104 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bc88ac05-6199-4aa7-8650-5ca88793a660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:07.951552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1428/2854] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:07.961542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:07.963165 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:08.026219 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62aa031-ea20-4448-98c8-52ddf5e0fd1f admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:08.030739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f6aa0-225c-4cf8-a985-dc343b0313ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:08.031393 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f6aa0-225c-4cf8-a985-dc343b0313ca service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.031682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f6aa0-225c-4cf8-a985-dc343b0313ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.031962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f6aa0-225c-4cf8-a985-dc343b0313ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.041013 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f6aa0-225c-4cf8-a985-dc343b0313ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:08.042416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f6aa0-225c-4cf8-a985-dc343b0313ca service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:08.051714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1427/2855] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:08.320710 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88ac05-6199-4aa7-8650-5ca88793a660 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:08.322106 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bc88ac05-6199-4aa7-8650-5ca88793a660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:08.335036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88ac05-6199-4aa7-8650-5ca88793a660 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:08.340026 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:08.341008 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.341878 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.342496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] PATH_INFO: `/v3/projects/75294eafddc547caaaac2be9fa215bf2` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.343727 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75294eafddc547caaaac2be9fa215bf2)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:08.345849 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:08.391292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75294eafddc547caaaac2be9fa215bf2', 'request_id': 'req-5712096d-cd59-4774-a1e9-1294e457dea1'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:08.399984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75294eafddc547caaaac2be9fa215bf2', 'request_id': 'req-5712096d-cd59-4774-a1e9-1294e457dea1'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:08.410122 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75294eafddc547caaaac2be9fa215bf2', 'request_id': 'req-5712096d-cd59-4774-a1e9-1294e457dea1'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:08.421715 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] The token cache is being invalidate because project 75294eafddc547caaaac2be9fa215bf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:08.422025 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:08.422663 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:08.424198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1429/2856] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:08 2022] DELETE /identity/v3/projects/75294eafddc547caaaac2be9fa215bf2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:08.428253 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f6aa0-225c-4cf8-a985-dc343b0313ca service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:08.437689 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f6aa0-225c-4cf8-a985-dc343b0313ca service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:08.495763 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:08.496643 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.496922 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.497198 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] PATH_INFO: `/v3/users/7ad5a8ace9f9498db894a06ec4638506` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.498427 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ad5a8ace9f9498db894a06ec4638506)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:08.500321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:08.550696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:08.557660 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ad5a8ace9f9498db894a06ec4638506'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:08.571161 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ad5a8ace9f9498db894a06ec4638506'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:08.578528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ad5a8ace9f9498db894a06ec4638506'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:08.579038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ad5a8ace9f9498db894a06ec4638506'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:08.586647 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ad5a8ace9f9498db894a06ec4638506'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:08.592952 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1428/2857] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:08 2022] DELETE /identity/v3/users/7ad5a8ace9f9498db894a06ec4638506 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:08.874456 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:08.876067 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:08.895091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712096d-cd59-4774-a1e9-1294e457dea1 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:08.900521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:08.953808 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:08.954699 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.954978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.955250 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] PATH_INFO: `/v3/projects/68415b9d92fb44759ef4dc7584eb70d5` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.956411 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68415b9d92fb44759ef4dc7584eb70d5)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:08.958906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:09.004580 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68415b9d92fb44759ef4dc7584eb70d5', 'request_id': 'req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.011505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68415b9d92fb44759ef4dc7584eb70d5', 'request_id': 'req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.017111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68415b9d92fb44759ef4dc7584eb70d5', 'request_id': 'req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.028308 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] The token cache is being invalidate because project 68415b9d92fb44759ef4dc7584eb70d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:09.028592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.029211 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.030762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1430/2858] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:08 2022] DELETE /identity/v3/projects/68415b9d92fb44759ef4dc7584eb70d5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:09.034744 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:09.036178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:09.049344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942528c-2286-41f4-a96b-8da3d1a1c049 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:09.104323 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4dfc7-f82b-42c2-bf43-6dd13c73208b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:09.105119 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4dfc7-f82b-42c2-bf43-6dd13c73208b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.105403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4dfc7-f82b-42c2-bf43-6dd13c73208b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.105678 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4dfc7-f82b-42c2-bf43-6dd13c73208b admin admin] PATH_INFO: `/v3/users/eaecddecaeda4c65ae3a1cbaa2f7a889` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.106895 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4dfc7-f82b-42c2-bf43-6dd13c73208b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaecddecaeda4c65ae3a1cbaa2f7a889)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:09.108809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4dfc7-f82b-42c2-bf43-6dd13c73208b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:09.154748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2db4dfc7-f82b-42c2-bf43-6dd13c73208b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaecddecaeda4c65ae3a1cbaa2f7a889'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.167220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2db4dfc7-f82b-42c2-bf43-6dd13c73208b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaecddecaeda4c65ae3a1cbaa2f7a889'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.173589 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2db4dfc7-f82b-42c2-bf43-6dd13c73208b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaecddecaeda4c65ae3a1cbaa2f7a889'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.174202 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2db4dfc7-f82b-42c2-bf43-6dd13c73208b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaecddecaeda4c65ae3a1cbaa2f7a889'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.181000 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-2db4dfc7-f82b-42c2-bf43-6dd13c73208b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaecddecaeda4c65ae3a1cbaa2f7a889'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.186798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1429/2859] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:09 2022] DELETE /identity/v3/users/eaecddecaeda4c65ae3a1cbaa2f7a889 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:09.441858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:09.443856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:09.457536 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6119e9-7cf6-4b4a-92aa-cf4479fef100 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:09.517066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:09.517932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.518213 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.518519 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] PATH_INFO: `/v3/projects/3492baf8273c4076b4489b62084781f9` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.519690 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3492baf8273c4076b4489b62084781f9)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:09.522249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:09.560097 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:09.570252 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3492baf8273c4076b4489b62084781f9', 'request_id': 'req-0429525d-d12a-450b-9b5e-86b216b7cad8'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.577837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3492baf8273c4076b4489b62084781f9', 'request_id': 'req-0429525d-d12a-450b-9b5e-86b216b7cad8'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.585955 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3492baf8273c4076b4489b62084781f9', 'request_id': 'req-0429525d-d12a-450b-9b5e-86b216b7cad8'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.597247 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] The token cache is being invalidate because project 3492baf8273c4076b4489b62084781f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:09.597547 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.598186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0429525d-d12a-450b-9b5e-86b216b7cad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:09.599782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1431/2860] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:09 2022] DELETE /identity/v3/projects/3492baf8273c4076b4489b62084781f9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:09.605663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd8bfc-e89c-4af2-8725-f4ee996aca76 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.605955 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd8bfc-e89c-4af2-8725-f4ee996aca76 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.606252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd8bfc-e89c-4af2-8725-f4ee996aca76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.623821 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-3abd8bfc-e89c-4af2-8725-f4ee996aca76 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:09.624946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3abd8bfc-e89c-4af2-8725-f4ee996aca76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:09.656946 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:09.686739 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1430/2861] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:09.695584 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19877ff-6379-42d8-b5bb-1bf17ad18940 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.695871 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19877ff-6379-42d8-b5bb-1bf17ad18940 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.696145 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19877ff-6379-42d8-b5bb-1bf17ad18940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.715697 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-f19877ff-6379-42d8-b5bb-1bf17ad18940 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:09.716929 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-f19877ff-6379-42d8-b5bb-1bf17ad18940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:09.724789 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1432/2862] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:09.728730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd8bfc-e89c-4af2-8725-f4ee996aca76 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:09.738093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd8bfc-e89c-4af2-8725-f4ee996aca76 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:09.742703 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7aa727-58b6-4b94-91ec-4b9f2a4d2c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:09.743353 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7aa727-58b6-4b94-91ec-4b9f2a4d2c19 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.743638 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7aa727-58b6-4b94-91ec-4b9f2a4d2c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.743921 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7aa727-58b6-4b94-91ec-4b9f2a4d2c19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.744645 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7aa727-58b6-4b94-91ec-4b9f2a4d2c19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:09.744955 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7aa727-58b6-4b94-91ec-4b9f2a4d2c19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:09.746571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7aa727-58b6-4b94-91ec-4b9f2a4d2c19 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:09.758339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1431/2863] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:09.762596 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19877ff-6379-42d8-b5bb-1bf17ad18940 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:09.772489 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19877ff-6379-42d8-b5bb-1bf17ad18940 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:09.777061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d078a-ae6e-45fd-b72f-6c38ca23753a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:09.777705 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d078a-ae6e-45fd-b72f-6c38ca23753a admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.777985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d078a-ae6e-45fd-b72f-6c38ca23753a admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.778258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d078a-ae6e-45fd-b72f-6c38ca23753a admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.778979 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d078a-ae6e-45fd-b72f-6c38ca23753a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:09.780883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d078a-ae6e-45fd-b72f-6c38ca23753a admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:09.806359 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1433/2864] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:42:09 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:09.809649 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7aa727-58b6-4b94-91ec-4b9f2a4d2c19 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:09.820191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7aa727-58b6-4b94-91ec-4b9f2a4d2c19 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:09.881170 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c79c8-7fb6-4126-93d2-6e0bfee40fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:09.881877 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c79c8-7fb6-4126-93d2-6e0bfee40fbd admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.882162 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c79c8-7fb6-4126-93d2-6e0bfee40fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.882476 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c79c8-7fb6-4126-93d2-6e0bfee40fbd admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.883189 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c79c8-7fb6-4126-93d2-6e0bfee40fbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:09.885208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c79c8-7fb6-4126-93d2-6e0bfee40fbd admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:09.914600 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1432/2865] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:42:09 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:09.916190 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d078a-ae6e-45fd-b72f-6c38ca23753a admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:09.927648 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d078a-ae6e-45fd-b72f-6c38ca23753a admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:09.932783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c8f7f-4c43-419f-817b-513b7a4b0c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:09.934985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8f7f-4c43-419f-817b-513b7a4b0c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.934985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8f7f-4c43-419f-817b-513b7a4b0c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.934985 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8f7f-4c43-419f-817b-513b7a4b0c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.935246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c8f7f-4c43-419f-817b-513b7a4b0c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:09.936673 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c8f7f-4c43-419f-817b-513b7a4b0c90 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:09.953432 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1434/2866] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:09.957397 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c79c8-7fb6-4126-93d2-6e0bfee40fbd admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:09.968241 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c79c8-7fb6-4126-93d2-6e0bfee40fbd admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:09.972941 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ed1c7-cff2-4c71-ab83-9ecb9b6e9f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:09.973602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ed1c7-cff2-4c71-ab83-9ecb9b6e9f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.973940 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ed1c7-cff2-4c71-ab83-9ecb9b6e9f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.974157 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ed1c7-cff2-4c71-ab83-9ecb9b6e9f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.974926 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ed1c7-cff2-4c71-ab83-9ecb9b6e9f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:09.976549 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ed1c7-cff2-4c71-ab83-9ecb9b6e9f92 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:09.987096 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1433/2867] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:09.991804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c8f7f-4c43-419f-817b-513b7a4b0c90 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:10.002123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c8f7f-4c43-419f-817b-513b7a4b0c90 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:10.007410 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e017d5-2a2c-4dd1-a0e9-c1c03c0b8cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:10.008230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e017d5-2a2c-4dd1-a0e9-c1c03c0b8cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:10.008513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e017d5-2a2c-4dd1-a0e9-c1c03c0b8cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:10.008791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e017d5-2a2c-4dd1-a0e9-c1c03c0b8cdd admin admin] PATH_INFO: `/v3/projects/45378ad1cf054f2e9599bd2e4c82000b/users/ec4a6bb77d3e4bca82e43f297c8897d5/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:10.023010 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e017d5-2a2c-4dd1-a0e9-c1c03c0b8cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=45378ad1cf054f2e9599bd2e4c82000b, user_id=ec4a6bb77d3e4bca82e43f297c8897d5, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:10.025669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e017d5-2a2c-4dd1-a0e9-c1c03c0b8cdd admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:10.039970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1435/2868] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:09 2022] PUT /identity/v3/projects/45378ad1cf054f2e9599bd2e4c82000b/users/ec4a6bb77d3e4bca82e43f297c8897d5/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:10.051325 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696e019-90ec-4114-b7d0-2aaa9c301e1b None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:10.052369 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696e019-90ec-4114-b7d0-2aaa9c301e1b None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:10.052467 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696e019-90ec-4114-b7d0-2aaa9c301e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:10.081335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-4696e019-90ec-4114-b7d0-2aaa9c301e1b None None] MFA Rules not processed for user `ec4a6bb77d3e4bca82e43f297c8897d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:10.082576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4696e019-90ec-4114-b7d0-2aaa9c301e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:10.152288 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1434/2869] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:10.164385 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e017d5-2a2c-4dd1-a0e9-c1c03c0b8cdd admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:10.166204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-a0e017d5-2a2c-4dd1-a0e9-c1c03c0b8cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:10.227165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e017d5-2a2c-4dd1-a0e9-c1c03c0b8cdd admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:10.230923 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976933e4-0eec-4750-a4e9-1d7dc09fc765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:10.231577 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976933e4-0eec-4750-a4e9-1d7dc09fc765 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:10.231863 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976933e4-0eec-4750-a4e9-1d7dc09fc765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:10.232149 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976933e4-0eec-4750-a4e9-1d7dc09fc765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:10.242692 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976933e4-0eec-4750-a4e9-1d7dc09fc765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:10.245058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976933e4-0eec-4750-a4e9-1d7dc09fc765 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:10.254790 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1436/2870] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:11.308734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696e019-90ec-4114-b7d0-2aaa9c301e1b None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:11.318399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696e019-90ec-4114-b7d0-2aaa9c301e1b None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:11.321893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bc616-00c5-4986-9f32-90a1ac314779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:11.322571 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc616-00c5-4986-9f32-90a1ac314779 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.322854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc616-00c5-4986-9f32-90a1ac314779 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.323134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc616-00c5-4986-9f32-90a1ac314779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.324865 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f57bc616-00c5-4986-9f32-90a1ac314779 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:11.355475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:11.387898 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bc616-00c5-4986-9f32-90a1ac314779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:11.389342 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bc616-00c5-4986-9f32-90a1ac314779 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:11.399386 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1435/2871] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:19.395036 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976933e4-0eec-4750-a4e9-1d7dc09fc765 service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.396407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-976933e4-0eec-4750-a4e9-1d7dc09fc765 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:19.463021 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976933e4-0eec-4750-a4e9-1d7dc09fc765 service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.468138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.469056 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.469335 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.469606 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] PATH_INFO: `/v3/users/8237fb1c731d4917b2b825fbc0aadcdf` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.470813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8237fb1c731d4917b2b825fbc0aadcdf)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.472738 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.478668 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:19.524117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8237fb1c731d4917b2b825fbc0aadcdf'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:19.531079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8237fb1c731d4917b2b825fbc0aadcdf'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:19.531684 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8237fb1c731d4917b2b825fbc0aadcdf'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:19.532284 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8237fb1c731d4917b2b825fbc0aadcdf'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:19.544429 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8237fb1c731d4917b2b825fbc0aadcdf'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:19.556499 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1437/2872] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:19 2022] DELETE /identity/v3/users/8237fb1c731d4917b2b825fbc0aadcdf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:19.561152 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6f2d5-fdc1-48ab-aaf4-dc45d84e97a6 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.561663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6f2d5-fdc1-48ab-aaf4-dc45d84e97a6 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.561663 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6f2d5-fdc1-48ab-aaf4-dc45d84e97a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.578989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-a6c6f2d5-fdc1-48ab-aaf4-dc45d84e97a6 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:19.580197 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a6c6f2d5-fdc1-48ab-aaf4-dc45d84e97a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:19.637503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1436/2873] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:19.647409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.648782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:19.698614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6f2d5-fdc1-48ab-aaf4-dc45d84e97a6 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.699996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a6c6f2d5-fdc1-48ab-aaf4-dc45d84e97a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:19.712182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ed586-b0a9-4b42-ba7b-e18ab73aa900 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.712475 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6f2d5-fdc1-48ab-aaf4-dc45d84e97a6 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.715895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f7ff6-94c1-4f33-b8d3-36791e08ebd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.716551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f7ff6-94c1-4f33-b8d3-36791e08ebd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.716837 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f7ff6-94c1-4f33-b8d3-36791e08ebd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.717114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f7ff6-94c1-4f33-b8d3-36791e08ebd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.717209 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.718063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.718380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.718665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] PATH_INFO: `/v3/users/ec4a6bb77d3e4bca82e43f297c8897d5` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.719873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4a6bb77d3e4bca82e43f297c8897d5)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.721799 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.726750 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f7ff6-94c1-4f33-b8d3-36791e08ebd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.728189 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f7ff6-94c1-4f33-b8d3-36791e08ebd2 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.737733 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1438/2874] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:19.773203 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4a6bb77d3e4bca82e43f297c8897d5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:19.788553 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4a6bb77d3e4bca82e43f297c8897d5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:19.797301 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4a6bb77d3e4bca82e43f297c8897d5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:19.798213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4a6bb77d3e4bca82e43f297c8897d5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:19.808522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4a6bb77d3e4bca82e43f297c8897d5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:19.815487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1437/2875] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:19 2022] DELETE /identity/v3/users/ec4a6bb77d3e4bca82e43f297c8897d5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:42:19.820111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e45fb6-045e-418b-ae77-cfdea3c4c5db None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.820402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e45fb6-045e-418b-ae77-cfdea3c4c5db None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.820707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e45fb6-045e-418b-ae77-cfdea3c4c5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.838552 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-59e45fb6-045e-418b-ae77-cfdea3c4c5db None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:19.839754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-59e45fb6-045e-418b-ae77-cfdea3c4c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:19.902874 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1439/2876] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:19.913074 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.914602 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:19.956716 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:19.973783 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3321b40-2530-49c4-80bf-8ea477c0a5a7 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.977378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18c96b-144b-4e65-99a7-2ab85c081ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.978024 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18c96b-144b-4e65-99a7-2ab85c081ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.978334 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18c96b-144b-4e65-99a7-2ab85c081ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.978614 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18c96b-144b-4e65-99a7-2ab85c081ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.987873 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18c96b-144b-4e65-99a7-2ab85c081ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.989430 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18c96b-144b-4e65-99a7-2ab85c081ca6 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.999229 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1438/2877] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:42:20.008546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e45fb6-045e-418b-ae77-cfdea3c4c5db None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.009996 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-59e45fb6-045e-418b-ae77-cfdea3c4c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:20.030496 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e45fb6-045e-418b-ae77-cfdea3c4c5db None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.036556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.037423 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.037749 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.037969 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] PATH_INFO: `/v3/projects/39462a6ef7e5432597f3f83f5916e09a` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.039451 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=39462a6ef7e5432597f3f83f5916e09a)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.041474 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.069534 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.094199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39462a6ef7e5432597f3f83f5916e09a', 'request_id': 'req-b0206f71-5cb5-404f-8393-ae80a8178d0b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.101877 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39462a6ef7e5432597f3f83f5916e09a', 'request_id': 'req-b0206f71-5cb5-404f-8393-ae80a8178d0b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.107860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39462a6ef7e5432597f3f83f5916e09a', 'request_id': 'req-b0206f71-5cb5-404f-8393-ae80a8178d0b'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.119246 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] The token cache is being invalidate because project 39462a6ef7e5432597f3f83f5916e09a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:20.119532 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.120172 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.122015 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1440/2878] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:20 2022] DELETE /identity/v3/projects/39462a6ef7e5432597f3f83f5916e09a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:20.125894 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18c96b-144b-4e65-99a7-2ab85c081ca6 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.136356 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18c96b-144b-4e65-99a7-2ab85c081ca6 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.199400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32f026-66b6-463e-9723-aaf38801579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.200281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32f026-66b6-463e-9723-aaf38801579b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.200564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32f026-66b6-463e-9723-aaf38801579b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.200842 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32f026-66b6-463e-9723-aaf38801579b admin admin] PATH_INFO: `/v3/users/6d27f7287fd44535a54f7b984ab1a7dd` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.202040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32f026-66b6-463e-9723-aaf38801579b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d27f7287fd44535a54f7b984ab1a7dd)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.204012 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32f026-66b6-463e-9723-aaf38801579b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.251492 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-1c32f026-66b6-463e-9723-aaf38801579b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d27f7287fd44535a54f7b984ab1a7dd'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.264734 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-1c32f026-66b6-463e-9723-aaf38801579b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d27f7287fd44535a54f7b984ab1a7dd'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.274148 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-1c32f026-66b6-463e-9723-aaf38801579b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d27f7287fd44535a54f7b984ab1a7dd'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.274853 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.274970 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-1c32f026-66b6-463e-9723-aaf38801579b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d27f7287fd44535a54f7b984ab1a7dd'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.276579 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:20.283088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-1c32f026-66b6-463e-9723-aaf38801579b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d27f7287fd44535a54f7b984ab1a7dd'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.289926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0206f71-5cb5-404f-8393-ae80a8178d0b admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.290731 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1439/2879] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:20 2022] DELETE /identity/v3/users/6d27f7287fd44535a54f7b984ab1a7dd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:20.352919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.353934 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.354256 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.354572 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] PATH_INFO: `/v3/projects/45378ad1cf054f2e9599bd2e4c82000b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.355774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] RBAC: Authorizing `identity:delete_project(project_id=45378ad1cf054f2e9599bd2e4c82000b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.357906 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.408066 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45378ad1cf054f2e9599bd2e4c82000b', 'request_id': 'req-9a618fdf-e125-4e54-accd-7151d56f433c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.416804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45378ad1cf054f2e9599bd2e4c82000b', 'request_id': 'req-9a618fdf-e125-4e54-accd-7151d56f433c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.423013 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45378ad1cf054f2e9599bd2e4c82000b', 'request_id': 'req-9a618fdf-e125-4e54-accd-7151d56f433c'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.438876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] The token cache is being invalidate because project 45378ad1cf054f2e9599bd2e4c82000b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:20.439172 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.439858 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-9a618fdf-e125-4e54-accd-7151d56f433c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.441461 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1441/2880] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:20 2022] DELETE /identity/v3/projects/45378ad1cf054f2e9599bd2e4c82000b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:42:20.446780 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c649ad-73e2-46ed-87c5-c1b06c69c232 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.447076 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c649ad-73e2-46ed-87c5-c1b06c69c232 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.447372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c649ad-73e2-46ed-87c5-c1b06c69c232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.466682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-21c649ad-73e2-46ed-87c5-c1b06c69c232 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:20.467903 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-21c649ad-73e2-46ed-87c5-c1b06c69c232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:20.520949 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.530598 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1440/2881] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:20.538309 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab8443-cbdc-4ed7-888d-f4bf1d592bd4 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.538892 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab8443-cbdc-4ed7-888d-f4bf1d592bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.539214 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab8443-cbdc-4ed7-888d-f4bf1d592bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.560362 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-38ab8443-cbdc-4ed7-888d-f4bf1d592bd4 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:20.562166 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-38ab8443-cbdc-4ed7-888d-f4bf1d592bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:20.571924 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1442/2882] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:20.572093 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c649ad-73e2-46ed-87c5-c1b06c69c232 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.573064 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-21c649ad-73e2-46ed-87c5-c1b06c69c232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:20.576402 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab8443-cbdc-4ed7-888d-f4bf1d592bd4 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.585416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c649ad-73e2-46ed-87c5-c1b06c69c232 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.590863 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.591674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab8443-cbdc-4ed7-888d-f4bf1d592bd4 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.591810 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.592008 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.592287 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] PATH_INFO: `/v3/projects/32c53bafea0149ec9258d9d0d6e3de7f` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.593572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] RBAC: Authorizing `identity:delete_project(project_id=32c53bafea0149ec9258d9d0d6e3de7f)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.595536 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.606903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807d9ae-e4dc-4286-8659-336e2906a574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.607585 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807d9ae-e4dc-4286-8659-336e2906a574 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.607926 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807d9ae-e4dc-4286-8659-336e2906a574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.608094 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807d9ae-e4dc-4286-8659-336e2906a574 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.608833 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807d9ae-e4dc-4286-8659-336e2906a574 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:20.609136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807d9ae-e4dc-4286-8659-336e2906a574 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.610777 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807d9ae-e4dc-4286-8659-336e2906a574 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.622961 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1443/2883] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:20.627280 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807d9ae-e4dc-4286-8659-336e2906a574 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.638662 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807d9ae-e4dc-4286-8659-336e2906a574 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.650127 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32c53bafea0149ec9258d9d0d6e3de7f', 'request_id': 'req-5d97d59d-c25d-4355-aa98-59127ec3b31e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.657436 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.657876 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32c53bafea0149ec9258d9d0d6e3de7f', 'request_id': 'req-5d97d59d-c25d-4355-aa98-59127ec3b31e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.664761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32c53bafea0149ec9258d9d0d6e3de7f', 'request_id': 'req-5d97d59d-c25d-4355-aa98-59127ec3b31e'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.677441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] The token cache is being invalidate because project 32c53bafea0149ec9258d9d0d6e3de7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:20.677772 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.678508 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-5d97d59d-c25d-4355-aa98-59127ec3b31e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:20.680079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1441/2884] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:20 2022] DELETE /identity/v3/projects/32c53bafea0149ec9258d9d0d6e3de7f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:20.685915 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658e0d3-03fa-4d46-976b-bb74184efc3d None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.686210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658e0d3-03fa-4d46-976b-bb74184efc3d None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.686542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658e0d3-03fa-4d46-976b-bb74184efc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.702559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf7c9b-c5dd-449e-b180-3cffa551ee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.703224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf7c9b-c5dd-449e-b180-3cffa551ee7f admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.703615 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf7c9b-c5dd-449e-b180-3cffa551ee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.703898 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf7c9b-c5dd-449e-b180-3cffa551ee7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.704226 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-d658e0d3-03fa-4d46-976b-bb74184efc3d None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:20.704614 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf7c9b-c5dd-449e-b180-3cffa551ee7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.705416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d658e0d3-03fa-4d46-976b-bb74184efc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:20.706551 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf7c9b-c5dd-449e-b180-3cffa551ee7f admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.713693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1442/2885] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:42:20.723227 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bd493-7ff4-4350-8a36-94842d91eafe None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.724130 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bd493-7ff4-4350-8a36-94842d91eafe None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.724416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bd493-7ff4-4350-8a36-94842d91eafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.734185 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1444/2886] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:42:20 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:42:20.738120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf7c9b-c5dd-449e-b180-3cffa551ee7f admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.740327 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0dcf7c9b-c5dd-449e-b180-3cffa551ee7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:20.743503 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-561bd493-7ff4-4350-8a36-94842d91eafe None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:20.744682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-561bd493-7ff4-4350-8a36-94842d91eafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:20.803318 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf7c9b-c5dd-449e-b180-3cffa551ee7f admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.804073 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1443/2887] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:20.807984 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27823187-d756-416c-9a63-f6a21ac45abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.808188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bd493-7ff4-4350-8a36-94842d91eafe None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.808649 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27823187-d756-416c-9a63-f6a21ac45abf admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.808932 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27823187-d756-416c-9a63-f6a21ac45abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.809207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27823187-d756-416c-9a63-f6a21ac45abf admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.809901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27823187-d756-416c-9a63-f6a21ac45abf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.811865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27823187-d756-416c-9a63-f6a21ac45abf admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.817225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bd493-7ff4-4350-8a36-94842d91eafe None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.822188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b012d5-4668-488a-814b-4a2a678197b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.822884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b012d5-4668-488a-814b-4a2a678197b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.823173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b012d5-4668-488a-814b-4a2a678197b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.823453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b012d5-4668-488a-814b-4a2a678197b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.824208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b012d5-4668-488a-814b-4a2a678197b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:20.824520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b012d5-4668-488a-814b-4a2a678197b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.826104 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b012d5-4668-488a-814b-4a2a678197b8 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.832273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1445/2888] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:42:20 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:20.836340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27823187-d756-416c-9a63-f6a21ac45abf admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.838100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1444/2889] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:20.842252 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b012d5-4668-488a-814b-4a2a678197b8 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.846471 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27823187-d756-416c-9a63-f6a21ac45abf admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.851693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd3f11-b9f6-45e7-83d9-1220049829af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.852097 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b012d5-4668-488a-814b-4a2a678197b8 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.852345 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd3f11-b9f6-45e7-83d9-1220049829af admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.852628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd3f11-b9f6-45e7-83d9-1220049829af admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.852905 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd3f11-b9f6-45e7-83d9-1220049829af admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.853621 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd3f11-b9f6-45e7-83d9-1220049829af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.855248 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd3f11-b9f6-45e7-83d9-1220049829af admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.856755 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15353ccc-ddc1-43dd-92a1-e7f0f4b924a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.857390 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15353ccc-ddc1-43dd-92a1-e7f0f4b924a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.857686 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15353ccc-ddc1-43dd-92a1-e7f0f4b924a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.857959 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15353ccc-ddc1-43dd-92a1-e7f0f4b924a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.858684 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15353ccc-ddc1-43dd-92a1-e7f0f4b924a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.860576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15353ccc-ddc1-43dd-92a1-e7f0f4b924a4 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.865989 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1446/2890] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:42:20.869902 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd3f11-b9f6-45e7-83d9-1220049829af admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.881126 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd3f11-b9f6-45e7-83d9-1220049829af admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.885937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef56638-9011-475f-828a-151e487d264e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.886669 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef56638-9011-475f-828a-151e487d264e admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.886956 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef56638-9011-475f-828a-151e487d264e admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.887234 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef56638-9011-475f-828a-151e487d264e admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.888070 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef56638-9011-475f-828a-151e487d264e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.891634 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1445/2891] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:42:20 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:20.891758 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef56638-9011-475f-828a-151e487d264e admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.892665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15353ccc-ddc1-43dd-92a1-e7f0f4b924a4 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.903927 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1447/2892] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:42:20.905073 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef56638-9011-475f-828a-151e487d264e admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.915457 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef56638-9011-475f-828a-151e487d264e admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.938478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.956297 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15353ccc-ddc1-43dd-92a1-e7f0f4b924a4 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.960996 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d95725-8610-4010-971b-68725d6aad1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.961929 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95725-8610-4010-971b-68725d6aad1d admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.962211 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95725-8610-4010-971b-68725d6aad1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.962516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95725-8610-4010-971b-68725d6aad1d admin admin] PATH_INFO: `/v3/users` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.963204 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d95725-8610-4010-971b-68725d6aad1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.965110 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d95725-8610-4010-971b-68725d6aad1d admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.972884 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a3fb7-663d-4448-a610-8b185a86e3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.974433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3fb7-663d-4448-a610-8b185a86e3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.974717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3fb7-663d-4448-a610-8b185a86e3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.974991 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3fb7-663d-4448-a610-8b185a86e3d0 admin admin] PATH_INFO: `/v3/projects/f9b6fa96f0a8408dabc6cd8f37101e76/users/d1a6a5576865438ba750342cdec937dc/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.985234 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1446/2893] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:42:20 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:42:20.989893 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d95725-8610-4010-971b-68725d6aad1d admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.990850 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a3fb7-663d-4448-a610-8b185a86e3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9b6fa96f0a8408dabc6cd8f37101e76, user_id=d1a6a5576865438ba750342cdec937dc, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.993547 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a3fb7-663d-4448-a610-8b185a86e3d0 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.999720 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d95725-8610-4010-971b-68725d6aad1d admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.001970 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:21.005351 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6b400-5dd7-4335-9796-4f7529d76f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.006007 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6b400-5dd7-4335-9796-4f7529d76f06 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.006424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6b400-5dd7-4335-9796-4f7529d76f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.006639 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6b400-5dd7-4335-9796-4f7529d76f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.007338 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6b400-5dd7-4335-9796-4f7529d76f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.008195 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1448/2894] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:20 2022] PUT /identity/v3/projects/f9b6fa96f0a8408dabc6cd8f37101e76/users/d1a6a5576865438ba750342cdec937dc/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:42:21.009318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6b400-5dd7-4335-9796-4f7529d76f06 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.018529 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb5ee-f419-4023-91ab-1520c3c8a8dd None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.018814 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb5ee-f419-4023-91ab-1520c3c8a8dd None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.019090 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb5ee-f419-4023-91ab-1520c3c8a8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.025383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1447/2895] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:42:21.028967 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6b400-5dd7-4335-9796-4f7529d76f06 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.039388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6b400-5dd7-4335-9796-4f7529d76f06 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.045789 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-707cb5ee-f419-4023-91ab-1520c3c8a8dd None None] MFA Rules not processed for user `d1a6a5576865438ba750342cdec937dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:21.047083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-707cb5ee-f419-4023-91ab-1520c3c8a8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:21.095726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b8a74-c914-41f7-b444-fcdab027c2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.096399 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b8a74-c914-41f7-b444-fcdab027c2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.096696 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b8a74-c914-41f7-b444-fcdab027c2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.096978 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b8a74-c914-41f7-b444-fcdab027c2c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.097748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b8a74-c914-41f7-b444-fcdab027c2c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.099924 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b8a74-c914-41f7-b444-fcdab027c2c1 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.109749 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1448/2896] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:42:21.112383 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1449/2897] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:42:21.113730 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b8a74-c914-41f7-b444-fcdab027c2c1 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.123461 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b8a74-c914-41f7-b444-fcdab027c2c1 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.123605 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cb5ee-f419-4023-91ab-1520c3c8a8dd None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.124866 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-707cb5ee-f419-4023-91ab-1520c3c8a8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:21.127976 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7a740-6d12-42cb-884e-fb26e8723e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.128854 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7a740-6d12-42cb-884e-fb26e8723e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.129135 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7a740-6d12-42cb-884e-fb26e8723e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.129412 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7a740-6d12-42cb-884e-fb26e8723e9c admin admin] PATH_INFO: `/v3/projects/7ae76df6ad404589b0b16ba43d844c48/users/18a4a93a0cef4b1299b6c520639fed61/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.144281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7a740-6d12-42cb-884e-fb26e8723e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ae76df6ad404589b0b16ba43d844c48, user_id=18a4a93a0cef4b1299b6c520639fed61, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.147597 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7a740-6d12-42cb-884e-fb26e8723e9c admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.161767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1449/2898] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:21 2022] PUT /identity/v3/projects/7ae76df6ad404589b0b16ba43d844c48/users/18a4a93a0cef4b1299b6c520639fed61/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:42:21.166803 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e5e9e-184c-4446-ae8c-b686f6218fe9 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.167035 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e5e9e-184c-4446-ae8c-b686f6218fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.167310 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e5e9e-184c-4446-ae8c-b686f6218fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.182131 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cb5ee-f419-4023-91ab-1520c3c8a8dd None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.184981 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-fc5e5e9e-184c-4446-ae8c-b686f6218fe9 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:21.186168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-fc5e5e9e-184c-4446-ae8c-b686f6218fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:21.231223 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917d771-ea9d-425e-9615-9c84460f0e51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.232918 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917d771-ea9d-425e-9615-9c84460f0e51 service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.233204 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917d771-ea9d-425e-9615-9c84460f0e51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.233795 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917d771-ea9d-425e-9615-9c84460f0e51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.245383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1450/2899] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:21.254836 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e5e9e-184c-4446-ae8c-b686f6218fe9 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.256368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-fc5e5e9e-184c-4446-ae8c-b686f6218fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:21.291737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:21.293404 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917d771-ea9d-425e-9615-9c84460f0e51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.294904 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917d771-ea9d-425e-9615-9c84460f0e51 service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.298559 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=100 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:21.305324 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1450/2900] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:42:21.315861 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e5e9e-184c-4446-ae8c-b686f6218fe9 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.319528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdea78-3a7a-446e-906a-d94344c88acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.320173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdea78-3a7a-446e-906a-d94344c88acd service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.320453 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdea78-3a7a-446e-906a-d94344c88acd service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.320725 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdea78-3a7a-446e-906a-d94344c88acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.329692 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdea78-3a7a-446e-906a-d94344c88acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.331137 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdea78-3a7a-446e-906a-d94344c88acd service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.341217 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1451/2901] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:42:21.979289 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccafed-2401-4b80-bd9e-72d23efa5cdb None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.979821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccafed-2401-4b80-bd9e-72d23efa5cdb None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.980020 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccafed-2401-4b80-bd9e-72d23efa5cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.009152 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-1bccafed-2401-4b80-bd9e-72d23efa5cdb None None] MFA Rules not processed for user `18a4a93a0cef4b1299b6c520639fed61`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:22.010391 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-1bccafed-2401-4b80-bd9e-72d23efa5cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:22.080387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1451/2902] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:22.092403 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdea78-3a7a-446e-906a-d94344c88acd service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:22.102075 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdea78-3a7a-446e-906a-d94344c88acd service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:22.105705 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ffd52-219e-4290-80ea-7530dbeccdb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:22.106388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ffd52-219e-4290-80ea-7530dbeccdb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.106681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ffd52-219e-4290-80ea-7530dbeccdb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.106955 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ffd52-219e-4290-80ea-7530dbeccdb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.116088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ffd52-219e-4290-80ea-7530dbeccdb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:22.117520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ffd52-219e-4290-80ea-7530dbeccdb8 service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:22.126661 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1452/2903] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:23.463914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a881c-8971-4e6e-88db-76811b9860da None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:23.464400 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a881c-8971-4e6e-88db-76811b9860da None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:23.464505 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a881c-8971-4e6e-88db-76811b9860da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:23.482171 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-358a881c-8971-4e6e-88db-76811b9860da None None] MFA Rules not processed for user `18a4a93a0cef4b1299b6c520639fed61`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:23.483371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-358a881c-8971-4e6e-88db-76811b9860da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:23.490278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1452/2904] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:23.726856 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ffd52-219e-4290-80ea-7530dbeccdb8 service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:23.736225 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ffd52-219e-4290-80ea-7530dbeccdb8 service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:23.739913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f714d3-5a65-44a5-ba92-11171d8d2e03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:23.740546 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f714d3-5a65-44a5-ba92-11171d8d2e03 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:23.740827 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f714d3-5a65-44a5-ba92-11171d8d2e03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:23.741100 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f714d3-5a65-44a5-ba92-11171d8d2e03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:23.742880 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-08f714d3-5a65-44a5-ba92-11171d8d2e03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:23.752293 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f714d3-5a65-44a5-ba92-11171d8d2e03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:23.753714 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f714d3-5a65-44a5-ba92-11171d8d2e03 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:23.762884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1453/2905] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:24.310978 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a881c-8971-4e6e-88db-76811b9860da None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:24.323497 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a881c-8971-4e6e-88db-76811b9860da None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:24.329804 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:24.330920 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:24.331382 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:24.331644 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] PATH_INFO: `/v3/users/d1a6a5576865438ba750342cdec937dc` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:24.333060 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a6a5576865438ba750342cdec937dc)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:24.336980 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:24.398500 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a6a5576865438ba750342cdec937dc'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:24.407734 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a6a5576865438ba750342cdec937dc'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:24.408487 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a6a5576865438ba750342cdec937dc'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:24.408881 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a6a5576865438ba750342cdec937dc'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:24.425111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a6a5576865438ba750342cdec937dc'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:24.441249 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1453/2906] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:24 2022] DELETE /identity/v3/users/d1a6a5576865438ba750342cdec937dc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:24.445726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0697e0-f2d2-414b-842a-e82ac63cbc95 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:24.446046 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0697e0-f2d2-414b-842a-e82ac63cbc95 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:24.446368 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0697e0-f2d2-414b-842a-e82ac63cbc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:24.465294 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-1b0697e0-f2d2-414b-842a-e82ac63cbc95 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:24.466680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1b0697e0-f2d2-414b-842a-e82ac63cbc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:24.530775 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1454/2907] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:24.540993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:24.542405 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:24.601138 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d7531-fdbc-47e6-bb05-61fa5795ff3c admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:24.605018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398fb82-13c3-4771-85ff-49c3c4a0b623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:24.605676 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398fb82-13c3-4771-85ff-49c3c4a0b623 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:24.605959 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398fb82-13c3-4771-85ff-49c3c4a0b623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:24.606414 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398fb82-13c3-4771-85ff-49c3c4a0b623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:24.613919 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:24.617392 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398fb82-13c3-4771-85ff-49c3c4a0b623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:24.619174 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398fb82-13c3-4771-85ff-49c3c4a0b623 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:24.630671 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1454/2908] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:24.889923 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0697e0-f2d2-414b-842a-e82ac63cbc95 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:24.891595 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-1b0697e0-f2d2-414b-842a-e82ac63cbc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:24.904953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0697e0-f2d2-414b-842a-e82ac63cbc95 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:24.909818 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:24.910786 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:24.911680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:24.911961 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] PATH_INFO: `/v3/projects/f9b6fa96f0a8408dabc6cd8f37101e76` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:24.913214 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9b6fa96f0a8408dabc6cd8f37101e76)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:24.915400 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:24.973210 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9b6fa96f0a8408dabc6cd8f37101e76', 'request_id': 'req-af6102b9-0a74-4160-a2be-00be9a36ef10'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:24.993145 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9b6fa96f0a8408dabc6cd8f37101e76', 'request_id': 'req-af6102b9-0a74-4160-a2be-00be9a36ef10'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:24.999682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9b6fa96f0a8408dabc6cd8f37101e76', 'request_id': 'req-af6102b9-0a74-4160-a2be-00be9a36ef10'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:25.011374 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] The token cache is being invalidate because project f9b6fa96f0a8408dabc6cd8f37101e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:25.011665 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:25.012320 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-af6102b9-0a74-4160-a2be-00be9a36ef10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:25.018542 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1455/2909] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:24 2022] DELETE /identity/v3/projects/f9b6fa96f0a8408dabc6cd8f37101e76 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:25.020158 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59274d06-7dbb-4ad3-8158-309ab1e192be None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.021262 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59274d06-7dbb-4ad3-8158-309ab1e192be None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.021542 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59274d06-7dbb-4ad3-8158-309ab1e192be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.040453 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-59274d06-7dbb-4ad3-8158-309ab1e192be None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:25.042068 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-59274d06-7dbb-4ad3-8158-309ab1e192be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:25.114317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1455/2910] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:25.123603 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a638-4027-45fd-8317-be6225c32bbb None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.123688 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a638-4027-45fd-8317-be6225c32bbb None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.123931 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a638-4027-45fd-8317-be6225c32bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.149441 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-51c8a638-4027-45fd-8317-be6225c32bbb None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:25.150885 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-51c8a638-4027-45fd-8317-be6225c32bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:25.170256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1456/2911] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:25.174646 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59274d06-7dbb-4ad3-8158-309ab1e192be None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.186707 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59274d06-7dbb-4ad3-8158-309ab1e192be None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.191406 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09cbc0-2c89-4c34-9790-2c178d5f5087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.192080 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09cbc0-2c89-4c34-9790-2c178d5f5087 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.192895 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09cbc0-2c89-4c34-9790-2c178d5f5087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.193176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09cbc0-2c89-4c34-9790-2c178d5f5087 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.193914 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09cbc0-2c89-4c34-9790-2c178d5f5087 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:25.194224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09cbc0-2c89-4c34-9790-2c178d5f5087 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.195865 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09cbc0-2c89-4c34-9790-2c178d5f5087 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.209592 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1456/2912] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:25.213985 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8a638-4027-45fd-8317-be6225c32bbb None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.229107 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8a638-4027-45fd-8317-be6225c32bbb None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.233715 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ca75e-6a9a-4b66-8742-ac01d8650c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.234424 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ca75e-6a9a-4b66-8742-ac01d8650c87 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.234712 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ca75e-6a9a-4b66-8742-ac01d8650c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.234989 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ca75e-6a9a-4b66-8742-ac01d8650c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.235705 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ca75e-6a9a-4b66-8742-ac01d8650c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.237641 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ca75e-6a9a-4b66-8742-ac01d8650c87 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.271812 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1457/2913] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:42:25 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:25.272935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09cbc0-2c89-4c34-9790-2c178d5f5087 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.283079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09cbc0-2c89-4c34-9790-2c178d5f5087 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.355372 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80246735-2b5c-4c4a-a8d1-efe6448245b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.355949 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80246735-2b5c-4c4a-a8d1-efe6448245b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.356230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80246735-2b5c-4c4a-a8d1-efe6448245b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.356501 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80246735-2b5c-4c4a-a8d1-efe6448245b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.357232 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80246735-2b5c-4c4a-a8d1-efe6448245b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.359268 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80246735-2b5c-4c4a-a8d1-efe6448245b3 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.377794 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1457/2914] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:42:25 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:25.382528 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ca75e-6a9a-4b66-8742-ac01d8650c87 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.392429 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:25.397264 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ca75e-6a9a-4b66-8742-ac01d8650c87 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.408063 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a86abc-03f0-41ba-8829-0ea94084bbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.408741 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86abc-03f0-41ba-8829-0ea94084bbbc admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.409050 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86abc-03f0-41ba-8829-0ea94084bbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.409321 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86abc-03f0-41ba-8829-0ea94084bbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.410116 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a86abc-03f0-41ba-8829-0ea94084bbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.412193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a86abc-03f0-41ba-8829-0ea94084bbbc admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.423297 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1458/2915] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:25.427254 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80246735-2b5c-4c4a-a8d1-efe6448245b3 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.437183 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80246735-2b5c-4c4a-a8d1-efe6448245b3 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.441827 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35e525-7452-4800-a1b5-f80d4f0918f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.443032 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35e525-7452-4800-a1b5-f80d4f0918f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.443316 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35e525-7452-4800-a1b5-f80d4f0918f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.443591 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35e525-7452-4800-a1b5-f80d4f0918f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.444317 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35e525-7452-4800-a1b5-f80d4f0918f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.445913 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35e525-7452-4800-a1b5-f80d4f0918f2 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.456860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1458/2916] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:25.461220 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a86abc-03f0-41ba-8829-0ea94084bbbc admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.473874 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a86abc-03f0-41ba-8829-0ea94084bbbc admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.478531 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3bb17-f976-467e-be39-1027114ac88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.479416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3bb17-f976-467e-be39-1027114ac88b admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.479698 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3bb17-f976-467e-be39-1027114ac88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.479974 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3bb17-f976-467e-be39-1027114ac88b admin admin] PATH_INFO: `/v3/projects/503c875f305542c589a59ccc6a51cee5/users/4bddbb94ac734251a008a82a51614617/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.495359 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3bb17-f976-467e-be39-1027114ac88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=503c875f305542c589a59ccc6a51cee5, user_id=4bddbb94ac734251a008a82a51614617, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.498011 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3bb17-f976-467e-be39-1027114ac88b admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.511402 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1459/2917] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:42:25 2022] PUT /identity/v3/projects/503c875f305542c589a59ccc6a51cee5/users/4bddbb94ac734251a008a82a51614617/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:25.522024 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba43cc9-5f8b-416a-ad0b-d009854aae2e None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.522340 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba43cc9-5f8b-416a-ad0b-d009854aae2e None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.522623 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba43cc9-5f8b-416a-ad0b-d009854aae2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.548917 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-7ba43cc9-5f8b-416a-ad0b-d009854aae2e None None] MFA Rules not processed for user `4bddbb94ac734251a008a82a51614617`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:25.550123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-7ba43cc9-5f8b-416a-ad0b-d009854aae2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:25.601292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:25.622109 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1459/2918] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:25.633703 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3bb17-f976-467e-be39-1027114ac88b admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.635440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-4bf3bb17-f976-467e-be39-1027114ac88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:25.694676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3bb17-f976-467e-be39-1027114ac88b admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.701485 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbcc9c-db14-4934-b18b-cda022657c1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.702144 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbcc9c-db14-4934-b18b-cda022657c1c service cinder] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.702460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbcc9c-db14-4934-b18b-cda022657c1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.702737 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbcc9c-db14-4934-b18b-cda022657c1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.712191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbcc9c-db14-4934-b18b-cda022657c1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.713618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbcc9c-db14-4934-b18b-cda022657c1c service cinder] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.722680 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1460/2919] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:25.747578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba43cc9-5f8b-416a-ad0b-d009854aae2e None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.780343 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbcc9c-db14-4934-b18b-cda022657c1c service cinder] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.782691 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a6dbcc9c-db14-4934-b18b-cda022657c1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:25.813187 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba43cc9-5f8b-416a-ad0b-d009854aae2e None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.819394 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.820346 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.820635 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.820995 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] PATH_INFO: `/v3/users/4bddbb94ac734251a008a82a51614617` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.822156 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bddbb94ac734251a008a82a51614617)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.824314 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.843882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbcc9c-db14-4934-b18b-cda022657c1c service cinder] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.847798 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033fcc4-6726-4ec7-a0b9-e0a58b52785b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.848522 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033fcc4-6726-4ec7-a0b9-e0a58b52785b service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.848882 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033fcc4-6726-4ec7-a0b9-e0a58b52785b service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.849233 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033fcc4-6726-4ec7-a0b9-e0a58b52785b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.851384 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-7033fcc4-6726-4ec7-a0b9-e0a58b52785b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:25.862125 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033fcc4-6726-4ec7-a0b9-e0a58b52785b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.863618 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033fcc4-6726-4ec7-a0b9-e0a58b52785b service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.871598 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bddbb94ac734251a008a82a51614617'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:25.872567 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1461/2920] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:42:25.878986 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bddbb94ac734251a008a82a51614617'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:25.879634 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bddbb94ac734251a008a82a51614617'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:25.880826 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bddbb94ac734251a008a82a51614617'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:25.899603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bddbb94ac734251a008a82a51614617'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:25.912931 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1460/2921] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:42:25 2022] DELETE /identity/v3/users/4bddbb94ac734251a008a82a51614617 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:25.917855 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a61d8-1ec8-4cd9-b47b-6ed425a76f47 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.918355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a61d8-1ec8-4cd9-b47b-6ed425a76f47 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.918472 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a61d8-1ec8-4cd9-b47b-6ed425a76f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.937918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-3e6a61d8-1ec8-4cd9-b47b-6ed425a76f47 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:25.939190 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3e6a61d8-1ec8-4cd9-b47b-6ed425a76f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:25.998416 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1462/2922] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:42:26.008000 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:26.009435 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:26.069207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af6fa9-6ede-461f-aea4-7fbf2e6eaeca admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:26.073176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c4489-0a81-4682-83e4-0e357829be80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:26.073836 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c4489-0a81-4682-83e4-0e357829be80 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.074125 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c4489-0a81-4682-83e4-0e357829be80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.074441 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c4489-0a81-4682-83e4-0e357829be80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.085034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c4489-0a81-4682-83e4-0e357829be80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:26.086964 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c4489-0a81-4682-83e4-0e357829be80 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:26.097007 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1461/2923] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:42:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:42:26.469987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a61d8-1ec8-4cd9-b47b-6ed425a76f47 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:26.471761 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3e6a61d8-1ec8-4cd9-b47b-6ed425a76f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:26.490846 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a61d8-1ec8-4cd9-b47b-6ed425a76f47 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:26.495682 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:26.496563 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.496832 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.497114 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] PATH_INFO: `/v3/projects/503c875f305542c589a59ccc6a51cee5` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.498406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] RBAC: Authorizing `identity:delete_project(project_id=503c875f305542c589a59ccc6a51cee5)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:26.500326 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:26.512213 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:26.554108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '503c875f305542c589a59ccc6a51cee5', 'request_id': 'req-62d4ad9f-afb5-46c8-a79e-9be0e856577a'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:26.566380 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '503c875f305542c589a59ccc6a51cee5', 'request_id': 'req-62d4ad9f-afb5-46c8-a79e-9be0e856577a'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:26.581642 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '503c875f305542c589a59ccc6a51cee5', 'request_id': 'req-62d4ad9f-afb5-46c8-a79e-9be0e856577a'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:26.596164 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] The token cache is being invalidate because project 503c875f305542c589a59ccc6a51cee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:42:26.596559 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:26.597383 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:42:26.599584 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1463/2924] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:42:26 2022] DELETE /identity/v3/projects/503c875f305542c589a59ccc6a51cee5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:36.092363 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:36.592313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:42.753626 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87c179-afef-4a3a-b7e6-a38764e85360 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:42.754114 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87c179-afef-4a3a-b7e6-a38764e85360 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:42.754207 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87c179-afef-4a3a-b7e6-a38764e85360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:42.782521 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-0f87c179-afef-4a3a-b7e6-a38764e85360 None None] MFA Rules not processed for user `18a4a93a0cef4b1299b6c520639fed61`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:42.783679 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0f87c179-afef-4a3a-b7e6-a38764e85360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:42.850683 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1462/2925] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:43:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:43:43.965778 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:43.967193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:44.090454 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4ad9f-afb5-46c8-a79e-9be0e856577a admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:44.095532 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f93e43-8ca5-4daa-9476-93f7cf834392 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:44.096181 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f93e43-8ca5-4daa-9476-93f7cf834392 service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:44.096468 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f93e43-8ca5-4daa-9476-93f7cf834392 service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:44.096745 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f93e43-8ca5-4daa-9476-93f7cf834392 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:44.098498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a0f93e43-8ca5-4daa-9476-93f7cf834392 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:44.110228 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f93e43-8ca5-4daa-9476-93f7cf834392 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:44.111816 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f93e43-8ca5-4daa-9476-93f7cf834392 service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:44.121465 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1464/2926] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:43:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:43:52.844083 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=35 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:54.117281 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=54 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:48.323737 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87c179-afef-4a3a-b7e6-a38764e85360 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:48.325236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-0f87c179-afef-4a3a-b7e6-a38764e85360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:48.425764 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87c179-afef-4a3a-b7e6-a38764e85360 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:48.430326 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299967b-dfa7-482c-ab66-ae611ae583fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:48.431201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299967b-dfa7-482c-ab66-ae611ae583fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:48.431483 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299967b-dfa7-482c-ab66-ae611ae583fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:48.431756 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299967b-dfa7-482c-ab66-ae611ae583fc admin admin] PATH_INFO: `/v3/users/18a4a93a0cef4b1299b6c520639fed61` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:48.432951 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299967b-dfa7-482c-ab66-ae611ae583fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a4a93a0cef4b1299b6c520639fed61)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:48.434925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299967b-dfa7-482c-ab66-ae611ae583fc admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:48.481018 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-e299967b-dfa7-482c-ab66-ae611ae583fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a4a93a0cef4b1299b6c520639fed61'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:44:48.488115 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-e299967b-dfa7-482c-ab66-ae611ae583fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a4a93a0cef4b1299b6c520639fed61'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:44:48.488729 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-e299967b-dfa7-482c-ab66-ae611ae583fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a4a93a0cef4b1299b6c520639fed61'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:44:48.489348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-e299967b-dfa7-482c-ab66-ae611ae583fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a4a93a0cef4b1299b6c520639fed61'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:44:48.501079 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-e299967b-dfa7-482c-ab66-ae611ae583fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a4a93a0cef4b1299b6c520639fed61'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:44:48.512860 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1463/2927] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:44:48 2022] DELETE /identity/v3/users/18a4a93a0cef4b1299b6c520639fed61 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:44:48.682193 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f93e43-8ca5-4daa-9476-93f7cf834392 service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:48.694173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f93e43-8ca5-4daa-9476-93f7cf834392 service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:48.752495 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:48.753372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:48.753681 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:48.753981 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] PATH_INFO: `/v3/projects/7ae76df6ad404589b0b16ba43d844c48` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:48.755208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ae76df6ad404589b0b16ba43d844c48)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:48.757123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:48.803108 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ae76df6ad404589b0b16ba43d844c48', 'request_id': 'req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:44:48.810256 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ae76df6ad404589b0b16ba43d844c48', 'request_id': 'req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:44:48.815890 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ae76df6ad404589b0b16ba43d844c48', 'request_id': 'req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:44:48.827344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] The token cache is being invalidate because project 7ae76df6ad404589b0b16ba43d844c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:44:48.827633 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:44:48.828272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6a1aa120-3ba8-45ce-bb8b-615c1f6a8fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:44:48.829804 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1465/2928] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:44:48 2022] DELETE /identity/v3/projects/7ae76df6ad404589b0b16ba43d844c48 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:44:48.835061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4399d-a95e-4b20-b68f-ad6e9fb31cb1 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:48.835353 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4399d-a95e-4b20-b68f-ad6e9fb31cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:48.835628 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4399d-a95e-4b20-b68f-ad6e9fb31cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:48.854718 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-64d4399d-a95e-4b20-b68f-ad6e9fb31cb1 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:44:48.855925 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-64d4399d-a95e-4b20-b68f-ad6e9fb31cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:48.900014 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:48.916765 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1464/2929] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:44:48.924849 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cae65-6b83-4b90-9b80-a6a666a245af None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:48.925201 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cae65-6b83-4b90-9b80-a6a666a245af None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:48.925435 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cae65-6b83-4b90-9b80-a6a666a245af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:48.943344 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-510cae65-6b83-4b90-9b80-a6a666a245af None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:44:48.944538 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-510cae65-6b83-4b90-9b80-a6a666a245af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:48.952904 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1466/2930] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:44:48.956803 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4399d-a95e-4b20-b68f-ad6e9fb31cb1 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:48.967292 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4399d-a95e-4b20-b68f-ad6e9fb31cb1 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:48.971856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f1daf-6487-4d80-a1b6-a316a8120108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:48.972504 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f1daf-6487-4d80-a1b6-a316a8120108 admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:48.972791 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f1daf-6487-4d80-a1b6-a316a8120108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:48.973101 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f1daf-6487-4d80-a1b6-a316a8120108 admin admin] PATH_INFO: `/v3/domains` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:48.973862 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f1daf-6487-4d80-a1b6-a316a8120108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:44:48.974165 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f1daf-6487-4d80-a1b6-a316a8120108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:48.975913 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f1daf-6487-4d80-a1b6-a316a8120108 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:48.987128 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1465/2931] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 13:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:48.991236 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510cae65-6b83-4b90-9b80-a6a666a245af None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:49.000355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510cae65-6b83-4b90-9b80-a6a666a245af None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:49.004962 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eba553-c2f0-438e-8690-07e57ea6683c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:49.005607 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eba553-c2f0-438e-8690-07e57ea6683c admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:49.005914 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eba553-c2f0-438e-8690-07e57ea6683c admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:49.006173 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eba553-c2f0-438e-8690-07e57ea6683c admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:49.006910 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eba553-c2f0-438e-8690-07e57ea6683c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:49.008796 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eba553-c2f0-438e-8690-07e57ea6683c admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:49.034352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1467/2932] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:44:48 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:44:49.038311 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f1daf-6487-4d80-a1b6-a316a8120108 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:49.048524 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f1daf-6487-4d80-a1b6-a316a8120108 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:49.104397 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dc649-9e9c-4b28-a429-4270a26beded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:49.105062 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dc649-9e9c-4b28-a429-4270a26beded admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:49.105347 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dc649-9e9c-4b28-a429-4270a26beded admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:49.105625 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dc649-9e9c-4b28-a429-4270a26beded admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:49.106348 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9dc649-9e9c-4b28-a429-4270a26beded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:49.108290 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9dc649-9e9c-4b28-a429-4270a26beded admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:49.128034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1466/2933] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:44:49 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:44:49.132028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eba553-c2f0-438e-8690-07e57ea6683c admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:49.142889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eba553-c2f0-438e-8690-07e57ea6683c admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:49.147834 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e2083-926f-43e0-bf43-3e5414acc35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:49.148774 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e2083-926f-43e0-bf43-3e5414acc35c admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:49.149005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e2083-926f-43e0-bf43-3e5414acc35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:49.149354 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e2083-926f-43e0-bf43-3e5414acc35c admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:49.150335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e2083-926f-43e0-bf43-3e5414acc35c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:49.152843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e2083-926f-43e0-bf43-3e5414acc35c admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:49.163302 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1468/2934] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:44:49.167882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dc649-9e9c-4b28-a429-4270a26beded admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:49.177782 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dc649-9e9c-4b28-a429-4270a26beded admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:49.182386 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90833f24-3fc2-40fa-9ac2-d756313da2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:49.183085 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90833f24-3fc2-40fa-9ac2-d756313da2da admin admin] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:49.183371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90833f24-3fc2-40fa-9ac2-d756313da2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:49.183650 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90833f24-3fc2-40fa-9ac2-d756313da2da admin admin] PATH_INFO: `/v3/roles` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:49.184371 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90833f24-3fc2-40fa-9ac2-d756313da2da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:49.185976 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90833f24-3fc2-40fa-9ac2-d756313da2da admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:49.195586 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1467/2935] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:44:49.200224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e2083-926f-43e0-bf43-3e5414acc35c admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:49.209998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e2083-926f-43e0-bf43-3e5414acc35c admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:49.215324 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3296cf3-f630-46e7-b7d1-4959e0c71c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:49.216191 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3296cf3-f630-46e7-b7d1-4959e0c71c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:49.216478 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3296cf3-f630-46e7-b7d1-4959e0c71c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:49.216753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3296cf3-f630-46e7-b7d1-4959e0c71c95 admin admin] PATH_INFO: `/v3/projects/efbe5203019c46b29f9c9f9bd9287a7d/users/e472fbc8eb174c1b991e6e402396fb56/roles/27d8d0ea0b83488a95a9141d439247f6` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:49.232178 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3296cf3-f630-46e7-b7d1-4959e0c71c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efbe5203019c46b29f9c9f9bd9287a7d, user_id=e472fbc8eb174c1b991e6e402396fb56, role_id=27d8d0ea0b83488a95a9141d439247f6)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:49.234870 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3296cf3-f630-46e7-b7d1-4959e0c71c95 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:49.247061 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1469/2936] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:44:49 2022] PUT /identity/v3/projects/efbe5203019c46b29f9c9f9bd9287a7d/users/e472fbc8eb174c1b991e6e402396fb56/roles/27d8d0ea0b83488a95a9141d439247f6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:44:49.251990 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853dd25-98e9-40e2-8a10-9e83dd91c5e8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:49.252283 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853dd25-98e9-40e2-8a10-9e83dd91c5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:49.252565 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853dd25-98e9-40e2-8a10-9e83dd91c5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:49.269856 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-8853dd25-98e9-40e2-8a10-9e83dd91c5e8 None None] MFA Rules not processed for user `c0168c20cbe64bb7a4d17219c4fe3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:44:49.271202 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-8853dd25-98e9-40e2-8a10-9e83dd91c5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:49.328224 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1468/2937] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:44:49.337555 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3296cf3-f630-46e7-b7d1-4959e0c71c95 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:49.340080 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-a3296cf3-f630-46e7-b7d1-4959e0c71c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:49.369809 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:49.410748 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3296cf3-f630-46e7-b7d1-4959e0c71c95 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:49.414747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5018f-72b9-45ae-af52-6351bd765b93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:49.415406 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5018f-72b9-45ae-af52-6351bd765b93 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:49.415690 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5018f-72b9-45ae-af52-6351bd765b93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:49.415970 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5018f-72b9-45ae-af52-6351bd765b93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:49.425516 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5018f-72b9-45ae-af52-6351bd765b93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:49.427033 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5018f-72b9-45ae-af52-6351bd765b93 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:49.436462 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1470/2938] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:44:53.458603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259a73-01a8-4273-99ea-56cca53fd7b8 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:53.459091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259a73-01a8-4273-99ea-56cca53fd7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:53.459091 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259a73-01a8-4273-99ea-56cca53fd7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:53.489502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-88259a73-01a8-4273-99ea-56cca53fd7b8 None None] MFA Rules not processed for user `e472fbc8eb174c1b991e6e402396fb56`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:44:53.490792 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-88259a73-01a8-4273-99ea-56cca53fd7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:53.541632 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:53.568113 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1469/2939] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:44:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:44:53.573052 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5018f-72b9-45ae-af52-6351bd765b93 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:53.584442 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5018f-72b9-45ae-af52-6351bd765b93 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:53.591510 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f883a-d0d2-4f56-bcb2-4c789efd0500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:53.592184 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f883a-d0d2-4f56-bcb2-4c789efd0500 admin admin] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:53.592469 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f883a-d0d2-4f56-bcb2-4c789efd0500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:53.592918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f883a-d0d2-4f56-bcb2-4c789efd0500 admin admin] PATH_INFO: `/v3/projects` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:53.593557 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f883a-d0d2-4f56-bcb2-4c789efd0500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:53.595498 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f883a-d0d2-4f56-bcb2-4c789efd0500 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:53.638376 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1471/2940] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 13:44:53 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:44:53.640428 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88259a73-01a8-4273-99ea-56cca53fd7b8 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:53.654140 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88259a73-01a8-4273-99ea-56cca53fd7b8 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:53.717613 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a394d-ff37-4d8d-8b97-8ecfb607dd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:53.718330 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a394d-ff37-4d8d-8b97-8ecfb607dd1d admin admin] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:53.718622 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a394d-ff37-4d8d-8b97-8ecfb607dd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:53.718901 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a394d-ff37-4d8d-8b97-8ecfb607dd1d admin admin] PATH_INFO: `/v3/users` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:53.719600 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a394d-ff37-4d8d-8b97-8ecfb607dd1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:53.721578 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a394d-ff37-4d8d-8b97-8ecfb607dd1d admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:53.743008 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1470/2941] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 13:44:53 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:44:53.747088 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f883a-d0d2-4f56-bcb2-4c789efd0500 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:53.757592 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f883a-d0d2-4f56-bcb2-4c789efd0500 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:53.762619 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795d804-cc58-4b08-9184-6904c2b8158c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:53.763299 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795d804-cc58-4b08-9184-6904c2b8158c admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:53.763582 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795d804-cc58-4b08-9184-6904c2b8158c admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:53.763886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795d804-cc58-4b08-9184-6904c2b8158c admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:53.764615 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795d804-cc58-4b08-9184-6904c2b8158c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:53.766217 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795d804-cc58-4b08-9184-6904c2b8158c admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:53.778249 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1472/2942] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:44:53.782513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a394d-ff37-4d8d-8b97-8ecfb607dd1d admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:53.792556 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a394d-ff37-4d8d-8b97-8ecfb607dd1d admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:53.798272 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a956ad-1209-4a64-aaaa-b6c4c7912135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:53.799215 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a956ad-1209-4a64-aaaa-b6c4c7912135 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:53.799533 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a956ad-1209-4a64-aaaa-b6c4c7912135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:53.799813 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a956ad-1209-4a64-aaaa-b6c4c7912135 admin admin] PATH_INFO: `/v3/projects/4aeda4161f514faca1a4f0e8ec795213/users/70ccddfe80364cce93871db3c59bd847/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:53.816078 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a956ad-1209-4a64-aaaa-b6c4c7912135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aeda4161f514faca1a4f0e8ec795213, user_id=70ccddfe80364cce93871db3c59bd847, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:53.818847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a956ad-1209-4a64-aaaa-b6c4c7912135 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:53.836545 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1471/2943] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:44:53 2022] PUT /identity/v3/projects/4aeda4161f514faca1a4f0e8ec795213/users/70ccddfe80364cce93871db3c59bd847/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:44:53.841318 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795d804-cc58-4b08-9184-6904c2b8158c admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:53.851782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795d804-cc58-4b08-9184-6904c2b8158c admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:53.917572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275352b-2156-4310-bedc-a3cc22eff5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:53.918410 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275352b-2156-4310-bedc-a3cc22eff5bf admin admin] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:53.918707 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275352b-2156-4310-bedc-a3cc22eff5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:53.918987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275352b-2156-4310-bedc-a3cc22eff5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:53.919726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275352b-2156-4310-bedc-a3cc22eff5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:53.921350 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275352b-2156-4310-bedc-a3cc22eff5bf admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:53.932188 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1473/2944] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 13:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:44:53.936396 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a956ad-1209-4a64-aaaa-b6c4c7912135 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:53.947322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a956ad-1209-4a64-aaaa-b6c4c7912135 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:53.953033 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91169495-706c-4b59-8aeb-e577205700a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:53.954186 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91169495-706c-4b59-8aeb-e577205700a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:53.954518 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91169495-706c-4b59-8aeb-e577205700a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:53.954797 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91169495-706c-4b59-8aeb-e577205700a8 admin admin] PATH_INFO: `/v3/projects/4aeda4161f514faca1a4f0e8ec795213/users/70ccddfe80364cce93871db3c59bd847/roles/0c473678b281495f86f58b077be04e1b` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:53.956883 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91169495-706c-4b59-8aeb-e577205700a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aeda4161f514faca1a4f0e8ec795213, user_id=70ccddfe80364cce93871db3c59bd847, role_id=0c473678b281495f86f58b077be04e1b)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:53.959746 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91169495-706c-4b59-8aeb-e577205700a8 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:53.972638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1472/2945] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 13:44:53 2022] PUT /identity/v3/projects/4aeda4161f514faca1a4f0e8ec795213/users/70ccddfe80364cce93871db3c59bd847/roles/0c473678b281495f86f58b077be04e1b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:44:58.272747 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab73ce-2f65-4124-bd01-e6a98b68e6d1 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:58.273183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab73ce-2f65-4124-bd01-e6a98b68e6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:58.273307 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab73ce-2f65-4124-bd01-e6a98b68e6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:58.304460 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-27ab73ce-2f65-4124-bd01-e6a98b68e6d1 None None] MFA Rules not processed for user `70ccddfe80364cce93871db3c59bd847`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:44:58.305676 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-27ab73ce-2f65-4124-bd01-e6a98b68e6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:58.383079 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1474/2946] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:44:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:44:58.400762 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91169495-706c-4b59-8aeb-e577205700a8 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:58.407847 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-91169495-706c-4b59-8aeb-e577205700a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:58.504291 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91169495-706c-4b59-8aeb-e577205700a8 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:58.509903 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc78529-fb81-4786-bef6-1d1a5c3247a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:58.510479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc78529-fb81-4786-bef6-1d1a5c3247a0 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:58.510766 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc78529-fb81-4786-bef6-1d1a5c3247a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:58.511043 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc78529-fb81-4786-bef6-1d1a5c3247a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:58.538075 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:58.571100 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc78529-fb81-4786-bef6-1d1a5c3247a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:58.572709 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc78529-fb81-4786-bef6-1d1a5c3247a0 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:58.582337 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1473/2947] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:44:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:44:59.618378 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf130b-b9b3-4634-bc3b-485f23375bc0 None None] REQUEST_METHOD: `POST` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:59.618886 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf130b-b9b3-4634-bc3b-485f23375bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:59.618977 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf130b-b9b3-4634-bc3b-485f23375bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:59.631392 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:59.638219 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.auth.core [None req-dccf130b-b9b3-4634-bc3b-485f23375bc0 None None] MFA Rules not processed for user `e472fbc8eb174c1b991e6e402396fb56`. Rule list: `[]` (Enabled: `True`). {{(pid=80091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:44:59.640038 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-dccf130b-b9b3-4634-bc3b-485f23375bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:59.647581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1475/2948] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 13:44:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:45:08.578198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=16 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:22.154950 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc78529-fb81-4786-bef6-1d1a5c3247a0 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:22.156513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-4bc78529-fb81-4786-bef6-1d1a5c3247a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:22.224322 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc78529-fb81-4786-bef6-1d1a5c3247a0 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:22.227941 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d53901-9df3-4ace-8dfb-e68f25d684ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c9f08515d514be8af1176717ec9e6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:22.228594 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d53901-9df3-4ace-8dfb-e68f25d684ed service cinder] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:22.228882 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d53901-9df3-4ace-8dfb-e68f25d684ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:22.229163 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d53901-9df3-4ace-8dfb-e68f25d684ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:22.230938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-15d53901-9df3-4ace-8dfb-e68f25d684ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:22.244560 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d53901-9df3-4ace-8dfb-e68f25d684ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:22.246059 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d53901-9df3-4ace-8dfb-e68f25d684ed service cinder] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:22.257469 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1474/2949] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:45:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:45:22.536878 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf130b-b9b3-4634-bc3b-485f23375bc0 None None] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:22.538352 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-dccf130b-b9b3-4634-bc3b-485f23375bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:22.594913 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf130b-b9b3-4634-bc3b-485f23375bc0 None None] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:22.598497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c370aec-6618-420a-ab0c-d0db72e48f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:22.599087 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c370aec-6618-420a-ab0c-d0db72e48f0b service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:22.599372 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c370aec-6618-420a-ab0c-d0db72e48f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:22.599650 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c370aec-6618-420a-ab0c-d0db72e48f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:22.601374 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-8c370aec-6618-420a-ab0c-d0db72e48f0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:22.611040 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c370aec-6618-420a-ab0c-d0db72e48f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:22.612497 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c370aec-6618-420a-ab0c-d0db72e48f0b service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:22.620365 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1476/2950] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:45:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:45:23.051147 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d53901-9df3-4ace-8dfb-e68f25d684ed service cinder] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:23.052525 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-15d53901-9df3-4ace-8dfb-e68f25d684ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:23.065059 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c370aec-6618-420a-ab0c-d0db72e48f0b service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:23.073572 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c370aec-6618-420a-ab0c-d0db72e48f0b service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:23.077174 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a50ef-2398-49b4-85fc-e6b6f211005b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:23.077807 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a50ef-2398-49b4-85fc-e6b6f211005b service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:23.078105 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a50ef-2398-49b4-85fc-e6b6f211005b service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:23.078388 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a50ef-2398-49b4-85fc-e6b6f211005b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:23.080224 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-324a50ef-2398-49b4-85fc-e6b6f211005b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:23.094837 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a50ef-2398-49b4-85fc-e6b6f211005b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:23.096266 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a50ef-2398-49b4-85fc-e6b6f211005b service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:23.108034 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1477/2951] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:45:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:45:23.123409 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d53901-9df3-4ace-8dfb-e68f25d684ed service cinder] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:23.128278 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a12fe6-0eb5-4a8f-afcf-e4423dee1d4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:23.128933 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a12fe6-0eb5-4a8f-afcf-e4423dee1d4f service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:23.129209 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a12fe6-0eb5-4a8f-afcf-e4423dee1d4f service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:23.129482 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a12fe6-0eb5-4a8f-afcf-e4423dee1d4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:23.131245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-d6a12fe6-0eb5-4a8f-afcf-e4423dee1d4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:23.143502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a12fe6-0eb5-4a8f-afcf-e4423dee1d4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:23.145111 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a12fe6-0eb5-4a8f-afcf-e4423dee1d4f service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:23.156236 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1475/2952] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:45:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:45:25.490655 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a50ef-2398-49b4-85fc-e6b6f211005b service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:25.492431 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-324a50ef-2398-49b4-85fc-e6b6f211005b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:25.508099 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a50ef-2398-49b4-85fc-e6b6f211005b service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:25.514394 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d7034-2c77-4018-8042-e21a9b43387d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:25.515134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7034-2c77-4018-8042-e21a9b43387d service nova] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:25.515487 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7034-2c77-4018-8042-e21a9b43387d service nova] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:25.515835 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7034-2c77-4018-8042-e21a9b43387d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:25.519103 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-2f4d7034-2c77-4018-8042-e21a9b43387d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:25.539517 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d7034-2c77-4018-8042-e21a9b43387d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:25.541183 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d7034-2c77-4018-8042-e21a9b43387d service nova] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:25.553918 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1478/2953] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:45:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:45:33.151273 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=63 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:35.066242 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a12fe6-0eb5-4a8f-afcf-e4423dee1d4f service placement] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:35.079767 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a12fe6-0eb5-4a8f-afcf-e4423dee1d4f service placement] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:35.085523 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6b5b9-c621-4efc-8bbb-5c1c228963f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:35.086182 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6b5b9-c621-4efc-8bbb-5c1c228963f4 service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:35.086494 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6b5b9-c621-4efc-8bbb-5c1c228963f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:35.086774 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6b5b9-c621-4efc-8bbb-5c1c228963f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:35.088657 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-e2e6b5b9-c621-4efc-8bbb-5c1c228963f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:35.102180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e6b5b9-c621-4efc-8bbb-5c1c228963f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:35.103786 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e6b5b9-c621-4efc-8bbb-5c1c228963f4 service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:35.115088 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1476/2954] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:45:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:45:35.548984 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=38 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:37.348366 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d7034-2c77-4018-8042-e21a9b43387d service nova] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:37.358134 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d7034-2c77-4018-8042-e21a9b43387d service nova] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:37.361776 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5525ae1-e8c4-4993-905c-b6b09b6777fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:37.362440 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5525ae1-e8c4-4993-905c-b6b09b6777fc service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:37.362726 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5525ae1-e8c4-4993-905c-b6b09b6777fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:37.362998 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5525ae1-e8c4-4993-905c-b6b09b6777fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:37.371753 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5525ae1-e8c4-4993-905c-b6b09b6777fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:37.373175 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5525ae1-e8c4-4993-905c-b6b09b6777fc service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:37.382444 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1479/2955] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:45:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:45:39.166784 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6b5b9-c621-4efc-8bbb-5c1c228963f4 service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:39.179120 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6b5b9-c621-4efc-8bbb-5c1c228963f4 service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:39.185022 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515040b1-8e8e-494b-88c6-30cf63b5a03a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b2edb6fcb7496bb55650080a9da172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:39.185754 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515040b1-8e8e-494b-88c6-30cf63b5a03a service nova] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:39.186112 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515040b1-8e8e-494b-88c6-30cf63b5a03a service nova] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:39.186502 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515040b1-8e8e-494b-88c6-30cf63b5a03a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:39.188407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-515040b1-8e8e-494b-88c6-30cf63b5a03a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:39.201405 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515040b1-8e8e-494b-88c6-30cf63b5a03a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:39.203149 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515040b1-8e8e-494b-88c6-30cf63b5a03a service nova] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:39.213798 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1477/2956] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:45:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:45:47.378290 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:49.209333 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=12 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:51.368670 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5525ae1-e8c4-4993-905c-b6b09b6777fc service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:51.397862 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5525ae1-e8c4-4993-905c-b6b09b6777fc service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:51.401574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34284d6-fb38-45a8-bad7-f586af369bd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:51.402218 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34284d6-fb38-45a8-bad7-f586af369bd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:51.402537 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34284d6-fb38-45a8-bad7-f586af369bd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:51.402814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34284d6-fb38-45a8-bad7-f586af369bd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:51.404581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-f34284d6-fb38-45a8-bad7-f586af369bd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:51.414389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34284d6-fb38-45a8-bad7-f586af369bd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:51.415814 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34284d6-fb38-45a8-bad7-f586af369bd6 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:51.423765 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1480/2957] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:47:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:01.420123 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=15 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:48:06.274141 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515040b1-8e8e-494b-88c6-30cf63b5a03a service nova] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:06.293034 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515040b1-8e8e-494b-88c6-30cf63b5a03a service nova] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:06.296601 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb193f48-82df-4244-88c5-734c24d870cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:06.297231 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb193f48-82df-4244-88c5-734c24d870cd service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:06.297510 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb193f48-82df-4244-88c5-734c24d870cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:06.297793 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb193f48-82df-4244-88c5-734c24d870cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:06.299546 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-bb193f48-82df-4244-88c5-734c24d870cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:06.308726 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb193f48-82df-4244-88c5-734c24d870cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:06.310199 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb193f48-82df-4244-88c5-734c24d870cd service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:06.318513 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1478/2958] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:48:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:06.414953 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34284d6-fb38-45a8-bad7-f586af369bd6 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:06.423699 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34284d6-fb38-45a8-bad7-f586af369bd6 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:06.427339 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c781576-cd50-40e1-8594-8b8108b67b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:06.427980 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c781576-cd50-40e1-8594-8b8108b67b87 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:06.428272 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c781576-cd50-40e1-8594-8b8108b67b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:06.428544 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c781576-cd50-40e1-8594-8b8108b67b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:06.430330 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-3c781576-cd50-40e1-8594-8b8108b67b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:06.440591 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c781576-cd50-40e1-8594-8b8108b67b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:06.442020 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c781576-cd50-40e1-8594-8b8108b67b87 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:06.450167 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1481/2959] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:48:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:06.571258 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb193f48-82df-4244-88c5-734c24d870cd service neutron] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:06.580180 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb193f48-82df-4244-88c5-734c24d870cd service neutron] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:06.583815 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2994e-a823-4fcc-862e-3c66e98447c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:06.584463 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2994e-a823-4fcc-862e-3c66e98447c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:06.584748 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2994e-a823-4fcc-862e-3c66e98447c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:06.585029 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2994e-a823-4fcc-862e-3c66e98447c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:06.586810 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-83c2994e-a823-4fcc-862e-3c66e98447c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:06.600829 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2994e-a823-4fcc-862e-3c66e98447c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:06.602270 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2994e-a823-4fcc-862e-3c66e98447c0 service neutron] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:06.613652 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1479/2960] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:48:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:16.446520 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:48:16.608305 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=17 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:48:51.103262 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c781576-cd50-40e1-8594-8b8108b67b87 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:51.115355 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c781576-cd50-40e1-8594-8b8108b67b87 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:51.120334 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3049945-5679-48ad-9143-3ee256b535c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:51.120987 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3049945-5679-48ad-9143-3ee256b535c6 service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:51.121273 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3049945-5679-48ad-9143-3ee256b535c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:51.121553 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3049945-5679-48ad-9143-3ee256b535c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:51.123357 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-d3049945-5679-48ad-9143-3ee256b535c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:51.135782 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3049945-5679-48ad-9143-3ee256b535c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:51.137387 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3049945-5679-48ad-9143-3ee256b535c6 service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:51.147938 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1482/2961] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:48:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:01.143028 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=6 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:49:48.607154 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21b95b-c5c7-461b-9836-e4975994e515 None None] REQUEST_METHOD: `POST` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:48.607674 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21b95b-c5c7-461b-9836-e4975994e515 None None] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:48.607776 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21b95b-c5c7-461b-9836-e4975994e515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:48.659935 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.auth.core [None req-de21b95b-c5c7-461b-9836-e4975994e515 None None] MFA Rules not processed for user `bb111be6bc2946d3b50b8d99a89cf8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:48.661058 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-de21b95b-c5c7-461b-9836-e4975994e515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:48.668176 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1480/2962] 10.222.0.45 () {58 vars in 1061 bytes} [Tue May 24 13:49:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:48.674083 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3049945-5679-48ad-9143-3ee256b535c6 service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:48.683041 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3049945-5679-48ad-9143-3ee256b535c6 service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:48.686540 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72f3bd-dd8b-4f26-9505-8b309eb17055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb111be6bc2946d3b50b8d99a89cf8ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:48.687168 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72f3bd-dd8b-4f26-9505-8b309eb17055 service neutron] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:48.687447 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72f3bd-dd8b-4f26-9505-8b309eb17055 service neutron] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:48.687719 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72f3bd-dd8b-4f26-9505-8b309eb17055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:48.748767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d72f3bd-dd8b-4f26-9505-8b309eb17055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:48.750208 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d72f3bd-dd8b-4f26-9505-8b309eb17055 service neutron] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:48.760298 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1483/2963] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:49:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:51.753351 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21b95b-c5c7-461b-9836-e4975994e515 None None] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:51.767035 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21b95b-c5c7-461b-9836-e4975994e515 None None] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:51.772086 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da03c4a-4e30-4b26-8110-64887af9ab4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:51.772851 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da03c4a-4e30-4b26-8110-64887af9ab4d service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:51.773421 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da03c4a-4e30-4b26-8110-64887af9ab4d service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:51.773627 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da03c4a-4e30-4b26-8110-64887af9ab4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:51.775658 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-3da03c4a-4e30-4b26-8110-64887af9ab4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:51.797387 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da03c4a-4e30-4b26-8110-64887af9ab4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:51.799051 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da03c4a-4e30-4b26-8110-64887af9ab4d service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:51.811230 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1481/2964] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:49:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:54.826742 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72f3bd-dd8b-4f26-9505-8b309eb17055 service neutron] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:54.828805 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-5d72f3bd-dd8b-4f26-9505-8b309eb17055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:54.848095 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72f3bd-dd8b-4f26-9505-8b309eb17055 service neutron] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:54.858146 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e878e-45a9-48c1-8b41-72096238bf28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:54.859009 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e878e-45a9-48c1-8b41-72096238bf28 service placement] REQUEST_METHOD: `GET` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:54.859295 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e878e-45a9-48c1-8b41-72096238bf28 service placement] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:54.859580 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e878e-45a9-48c1-8b41-72096238bf28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:54.861693 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-9c5e878e-45a9-48c1-8b41-72096238bf28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:54.885196 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e878e-45a9-48c1-8b41-72096238bf28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:54.886685 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e878e-45a9-48c1-8b41-72096238bf28 service placement] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:54.900973 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1484/2965] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:49:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:54.901937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da03c4a-4e30-4b26-8110-64887af9ab4d service placement] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:54.918264 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da03c4a-4e30-4b26-8110-64887af9ab4d service placement] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:54.923395 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:54.924433 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:54.924717 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:54.924993 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] PATH_INFO: `/v3/users/e472fbc8eb174c1b991e6e402396fb56` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:54.926744 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e472fbc8eb174c1b991e6e402396fb56)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:54.928783 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:54.978937 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e472fbc8eb174c1b991e6e402396fb56'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:54.985994 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e472fbc8eb174c1b991e6e402396fb56'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:54.986638 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e472fbc8eb174c1b991e6e402396fb56'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:54.987245 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e472fbc8eb174c1b991e6e402396fb56'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:54.999061 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e472fbc8eb174c1b991e6e402396fb56'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.012063 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1482/2966] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:49:54 2022] DELETE /identity/v3/users/e472fbc8eb174c1b991e6e402396fb56 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:49:55.183404 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e878e-45a9-48c1-8b41-72096238bf28 service placement] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:55.194729 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e878e-45a9-48c1-8b41-72096238bf28 service placement] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:55.254251 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:55.255142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:55.255426 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:55.255704 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] PATH_INFO: `/v3/projects/efbe5203019c46b29f9c9f9bd9287a7d` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:55.256869 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efbe5203019c46b29f9c9f9bd9287a7d)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:55.259389 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:55.303889 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efbe5203019c46b29f9c9f9bd9287a7d', 'request_id': 'req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.311005 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efbe5203019c46b29f9c9f9bd9287a7d', 'request_id': 'req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.316574 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efbe5203019c46b29f9c9f9bd9287a7d', 'request_id': 'req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.328025 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] The token cache is being invalidate because project efbe5203019c46b29f9c9f9bd9287a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:49:55.328313 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.328944 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.330564 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1485/2967] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:49:55 2022] DELETE /identity/v3/projects/efbe5203019c46b29f9c9f9bd9287a7d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:49:55.334479 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:55.335908 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:55.349117 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc95f7-39ff-4bb4-bf91-aaf04548da67 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:55.410483 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:55.411366 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:55.411708 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:55.411972 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] PATH_INFO: `/v3/users/70ccddfe80364cce93871db3c59bd847` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:55.413271 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70ccddfe80364cce93871db3c59bd847)` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:55.415769 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:55.455603 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:49:55.469821 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70ccddfe80364cce93871db3c59bd847'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.477201 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70ccddfe80364cce93871db3c59bd847'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.477819 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70ccddfe80364cce93871db3c59bd847'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.478448 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70ccddfe80364cce93871db3c59bd847'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.490121 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.notifications [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70ccddfe80364cce93871db3c59bd847'} {{(pid=80090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.502123 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1483/2968] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 13:49:55 2022] DELETE /identity/v3/users/70ccddfe80364cce93871db3c59bd847 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:49:55.679491 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] Authenticating user token {{(pid=80091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:55.680942 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.fernet_utils [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:55.696843 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af14bf2-0def-4341-8b91-fc8e7b8eb3b5 admin admin] Validating token access rules against request {{(pid=80091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:55.706576 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:49:55.758713 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0168c20cbe64bb7a4d17219c4fe3eef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26aa7dae9c0c4f5a892a6fcef95c15ea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:55.759581 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:55.759884 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:55.760142 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] PATH_INFO: `/v3/projects/4aeda4161f514faca1a4f0e8ec795213` {{(pid=80091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:55.761335 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aeda4161f514faca1a4f0e8ec795213)` {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:55.763304 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] RBAC: Authorization granted {{(pid=80091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:55.809432 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aeda4161f514faca1a4f0e8ec795213', 'request_id': 'req-86bdd76f-cff9-478d-9070-4f4e6ef2b854'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.816709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aeda4161f514faca1a4f0e8ec795213', 'request_id': 'req-86bdd76f-cff9-478d-9070-4f4e6ef2b854'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.823504 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aeda4161f514faca1a4f0e8ec795213', 'request_id': 'req-86bdd76f-cff9-478d-9070-4f4e6ef2b854'} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.835151 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] The token cache is being invalidate because project 4aeda4161f514faca1a4f0e8ec795213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:49:55.835767 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.836154 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG keystone.notifications [None req-86bdd76f-cff9-478d-9070-4f4e6ef2b854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:49:55.837709 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: [pid: 80091|app: 0|req: 1486/2969] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 13:49:55 2022] DELETE /identity/v3/projects/4aeda4161f514faca1a4f0e8ec795213 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:50:05.498693 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=17,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:50:05.832015 dv-cinder-843018-2-iscsi devstack@keystone.service[80091]: DEBUG dbcounter [-] [80091] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:53:54.065198 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] Authenticating user token {{(pid=80090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:53:54.066667 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:54.192407 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758c8fd-585f-4087-abeb-e4975083e456 admin admin] Validating token access rules against request {{(pid=80090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:53:54.248940 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2940-ae3b-4bcd-9c6b-becab8f67612 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '630f0527bd844de9be072c459dab96c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5887ec08efc449abba83b0c49e9f167', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:53:54.249587 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2940-ae3b-4bcd-9c6b-becab8f67612 service placement] REQUEST_METHOD: `GET` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:54.249873 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2940-ae3b-4bcd-9c6b-becab8f67612 service placement] SCRIPT_NAME: `/identity` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:54.250149 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2940-ae3b-4bcd-9c6b-becab8f67612 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:54.251938 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.fernet_utils [None req-6e6d2940-ae3b-4bcd-9c6b-becab8f67612 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:54.262876 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d2940-ae3b-4bcd-9c6b-becab8f67612 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:53:54.264466 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d2940-ae3b-4bcd-9c6b-becab8f67612 service placement] RBAC: Authorization granted {{(pid=80090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:53:54.274136 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: [pid: 80090|app: 0|req: 1484/2970] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 13:53:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:54:04.269982 dv-cinder-843018-2-iscsi devstack@keystone.service[80090]: DEBUG dbcounter [-] [80090] Writing DB stats keystone:SELECT=84 {{(pid=80090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}